Oct 30 12:54:15.542141 ip-10-0-142-84 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Oct 30 12:54:15.542155 ip-10-0-142-84 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Oct 30 12:54:15.542165 ip-10-0-142-84 systemd[1]: kubelet.service: Failed with result 'resources'.
Oct 30 12:54:15.542494 ip-10-0-142-84 systemd[1]: Failed to start Kubernetes Kubelet.
Oct 30 12:54:25.631227 ip-10-0-142-84 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Oct 30 12:54:25.631242 ip-10-0-142-84 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot a6bea42176b643568a080ebde2fd07c8 --
Oct 30 12:56:22.483522 ip-10-0-142-84 systemd[1]: Starting Kubernetes Kubelet...
Oct 30 12:56:22.876551 ip-10-0-142-84 kubenswrapper[2573]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 30 12:56:22.876551 ip-10-0-142-84 kubenswrapper[2573]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Oct 30 12:56:22.876551 ip-10-0-142-84 kubenswrapper[2573]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 30 12:56:22.876551 ip-10-0-142-84 kubenswrapper[2573]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 30 12:56:22.883034 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.882947    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 12:56:22.883034 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.883030    2573 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 12:56:22.883034 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.883034    2573 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 12:56:22.883034 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.883038    2573 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 12:56:22.883034 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.883042    2573 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 12:56:22.883227 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.883045    2573 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 12:56:22.883227 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.883049    2573 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 12:56:22.883227 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.883052    2573 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 12:56:22.883227 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.883055    2573 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 12:56:22.883227 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.883058    2573 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 12:56:22.883227 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.883061    2573 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 12:56:22.883227 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.883063    2573 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 12:56:22.883227 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.883066    2573 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 12:56:22.883227 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.883069    2573 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 12:56:22.883227 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.883072    2573 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 12:56:22.883227 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.883074    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 12:56:22.883227 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.883077    2573 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 12:56:22.883227 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.883080    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 12:56:22.883227 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.883083    2573 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 12:56:22.883227 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.883089    2573 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 12:56:22.883227 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.883092    2573 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 12:56:22.883227 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.883095    2573 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 12:56:22.883227 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.883097    2573 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 12:56:22.883227 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.883100    2573 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 12:56:22.883688 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.883103    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 12:56:22.883688 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.883105    2573 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 12:56:22.883688 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.883108    2573 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 12:56:22.883688 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.883111    2573 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 12:56:22.883688 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.883113    2573 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 12:56:22.883688 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.883116    2573 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 12:56:22.883688 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.883118    2573 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 12:56:22.883688 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.883121    2573 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 12:56:22.883688 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.883124    2573 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 12:56:22.883688 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.883126    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 12:56:22.883688 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.883129    2573 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 12:56:22.883688 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.883132    2573 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 12:56:22.883688 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.883134    2573 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 12:56:22.883688 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.883137    2573 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 12:56:22.883688 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.883140    2573 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 12:56:22.883688 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.883143    2573 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 12:56:22.883688 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.883146    2573 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 12:56:22.883688 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.883148    2573 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 12:56:22.883688 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.883151    2573 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 12:56:22.883688 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.883154    2573 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 12:56:22.884173 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.883156    2573 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 12:56:22.884173 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.883159    2573 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 12:56:22.884173 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.883162    2573 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 12:56:22.884173 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.883165    2573 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 12:56:22.884173 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.883167    2573 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 12:56:22.884173 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.883170    2573 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 12:56:22.884173 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.883173    2573 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 12:56:22.884173 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.883176    2573 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 12:56:22.884173 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.883179    2573 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 12:56:22.884173 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.883182    2573 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 12:56:22.884173 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.883184    2573 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 12:56:22.884173 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.883187    2573 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 12:56:22.884173 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.883190    2573 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 12:56:22.884173 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.883192    2573 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 12:56:22.884173 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.883195    2573 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 12:56:22.884173 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.883197    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 12:56:22.884173 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.883200    2573 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 12:56:22.884173 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.883203    2573 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 12:56:22.884173 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.883207    2573 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 12:56:22.884632 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.883212    2573 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 12:56:22.884632 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.883216    2573 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 12:56:22.884632 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.883219    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 12:56:22.884632 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.883222    2573 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 12:56:22.884632 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.883225    2573 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 12:56:22.884632 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.883227    2573 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 12:56:22.884632 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.883230    2573 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 12:56:22.884632 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.883235    2573 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 12:56:22.884632 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.883238    2573 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 12:56:22.884632 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.883241    2573 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 12:56:22.884632 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.883244    2573 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 12:56:22.884632 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.883247    2573 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 12:56:22.884632 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.883250    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 12:56:22.884632 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.883252    2573 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 12:56:22.884632 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.883255    2573 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 12:56:22.884632 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.883271    2573 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 12:56:22.884632 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.883274    2573 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 12:56:22.884632 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.883277    2573 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 12:56:22.884632 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.883280    2573 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 12:56:22.885086 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.883282    2573 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 12:56:22.885086 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.883286    2573 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 12:56:22.885086 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.883289    2573 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 12:56:22.885086 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.883292    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 12:56:22.885086 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.883295    2573 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 12:56:22.885086 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.883297    2573 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 12:56:22.885086 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.883301    2573 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 12:56:22.885086 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.883306    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 12:56:22.885086 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.883309    2573 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 12:56:22.885086 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.883311    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 12:56:22.885086 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.883314    2573 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 12:56:22.885086 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.883317    2573 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 12:56:22.885086 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.883319    2573 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 12:56:22.885086 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.883322    2573 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 12:56:22.885086 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.883325    2573 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 12:56:22.885086 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.883328    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 12:56:22.885086 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.883331    2573 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 12:56:22.885086 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.883333    2573 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 12:56:22.885086 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.883336    2573 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 12:56:22.885086 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.883339    2573 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 12:56:22.885605 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.883342    2573 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 12:56:22.885605 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.883345    2573 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 12:56:22.885605 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.883347    2573 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 12:56:22.885605 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.883350    2573 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 12:56:22.885605 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.884172    2573 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 12:56:22.885605 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.884180    2573 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 12:56:22.885605 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.884183    2573 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 12:56:22.885605 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.884186    2573 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 12:56:22.885605 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.884189    2573 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 12:56:22.885605 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.884192    2573 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 12:56:22.885605 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.884195    2573 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 12:56:22.885605 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.884198    2573 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 12:56:22.885605 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.884200    2573 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 12:56:22.885605 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.884205    2573 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 12:56:22.885605 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.884210    2573 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 12:56:22.885605 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.884214    2573 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 12:56:22.885605 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.884218    2573 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 12:56:22.885605 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.884221    2573 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 12:56:22.885605 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.884223    2573 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 12:56:22.885605 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.884227    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 12:56:22.886091 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.884230    2573 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 12:56:22.886091 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.884233    2573 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 12:56:22.886091 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.884235    2573 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 12:56:22.886091 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.884238    2573 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 12:56:22.886091 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.884241    2573 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 12:56:22.886091 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.884243    2573 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 12:56:22.886091 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.884246    2573 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 12:56:22.886091 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.884249    2573 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 12:56:22.886091 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.884252    2573 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 12:56:22.886091 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.884255    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 12:56:22.886091 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.884272    2573 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 12:56:22.886091 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.884275    2573 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 12:56:22.886091 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.884279    2573 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 12:56:22.886091 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.884282    2573 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 12:56:22.886091 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.884284    2573 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 12:56:22.886091 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.884287    2573 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 12:56:22.886091 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.884290    2573 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 12:56:22.886091 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.884293    2573 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 12:56:22.886091 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.884296    2573 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 12:56:22.886091 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.884298    2573 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 12:56:22.886584 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.884301    2573 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 12:56:22.886584 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.884304    2573 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 12:56:22.886584 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.884307    2573 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 12:56:22.886584 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.884309    2573 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 12:56:22.886584 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.884312    2573 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 12:56:22.886584 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.884315    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 12:56:22.886584 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.884318    2573 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 12:56:22.886584 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.884321    2573 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 12:56:22.886584 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.884323    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 12:56:22.886584 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.884326    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 12:56:22.886584 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.884329    2573 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 12:56:22.886584 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.884331    2573 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 12:56:22.886584 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.884334    2573 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 12:56:22.886584 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.884336    2573 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 12:56:22.886584 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.884339    2573 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 12:56:22.886584 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.884342    2573 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 12:56:22.886584 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.884344    2573 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 12:56:22.886584 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.884347    2573 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 12:56:22.886584 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.884350    2573 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 12:56:22.887026 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.884353    2573 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 12:56:22.887026 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.884356    2573 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 12:56:22.887026 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.884359    2573 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 12:56:22.887026 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.884361    2573 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 12:56:22.887026 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.884366    2573 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 12:56:22.887026 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.884369    2573 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 12:56:22.887026 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.884372    2573 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 12:56:22.887026 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.884375    2573 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 12:56:22.887026 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.884378    2573 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 12:56:22.887026 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.884381    2573 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 12:56:22.887026 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.884383    2573 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 12:56:22.887026 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.884386    2573 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 12:56:22.887026 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.884389    2573 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 12:56:22.887026 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.884391    2573 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 12:56:22.887026 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.884394    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 12:56:22.887026 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.884397    2573 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 12:56:22.887026 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.884399    2573 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 12:56:22.887026 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.884402    2573 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 12:56:22.887026 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.884404    2573 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 12:56:22.887488 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.884407    2573 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 12:56:22.887488 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.884410    2573 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 12:56:22.887488 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.884413    2573 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 12:56:22.887488 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.884416    2573 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 12:56:22.887488 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.884418    2573 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 12:56:22.887488 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.884421    2573 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 12:56:22.887488 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.884424    2573 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 12:56:22.887488 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.884426    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 12:56:22.887488 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.884429    2573 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 12:56:22.887488 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.884432    2573 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 12:56:22.887488 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.884434    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 12:56:22.887488 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.884437    2573 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 12:56:22.887488 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.884440    2573 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 12:56:22.887488 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.884443    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 12:56:22.887488 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.884446    2573 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 12:56:22.887488 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.884449    2573 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 12:56:22.887488 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.884451    2573 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 12:56:22.887488 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.884455    2573 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 12:56:22.887488 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.884458    2573 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 12:56:22.887488 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.884460    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 12:56:22.887953 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.884463    2573 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 12:56:22.887953 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.884466    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 12:56:22.887953 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.884468    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 12:56:22.887953 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.884473    2573 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 12:56:22.887953 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.884475    2573 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 12:56:22.887953 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.884478    2573 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 12:56:22.887953 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.884481    2573 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 12:56:22.887953 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.884483    2573 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 12:56:22.887953 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.884486    2573 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 12:56:22.887953 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.884489    2573 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 12:56:22.887953 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.884491    2573 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 12:56:22.887953 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.884494    2573 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 12:56:22.887953 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.884997    2573 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 12:56:22.887953 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.885002    2573 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 12:56:22.887953 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.885006    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 12:56:22.887953 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.885009    2573 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 12:56:22.887953 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.885013    2573 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 12:56:22.887953 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.885016    2573 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 12:56:22.887953 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.885018    2573 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 12:56:22.887953 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.885021    2573 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 12:56:22.888440 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.885024    2573 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 12:56:22.888440 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.885027    2573 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 12:56:22.888440 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.885030    2573 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 12:56:22.888440 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.885033    2573 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 12:56:22.888440 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.885036    2573 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 12:56:22.888440 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.885039    2573 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 12:56:22.888440 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.885042    2573 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 12:56:22.888440 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.885045    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 12:56:22.888440 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.885048    2573 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 12:56:22.888440 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.885050    2573 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 12:56:22.888440 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.885053    2573 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 12:56:22.888440 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.885056    2573 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 12:56:22.888440 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.885059    2573 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 12:56:22.888440 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.885061    2573 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 12:56:22.888440 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.885064    2573 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 12:56:22.888440 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.885067    2573 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 12:56:22.888440 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.885069    2573 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 12:56:22.888440 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.885072    2573 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 12:56:22.888440 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.885074    2573 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 12:56:22.888884 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.885077    2573 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 12:56:22.888884 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.885081    2573 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 12:56:22.888884 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.885084    2573 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 12:56:22.888884 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.885087    2573 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 12:56:22.888884 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.885090    2573 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 12:56:22.888884 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.885093    2573 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 12:56:22.888884 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.885095    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 12:56:22.888884 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.885099    2573 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 12:56:22.888884 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.885102    2573 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 12:56:22.888884 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.885105    2573 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 12:56:22.888884 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.885107    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 12:56:22.888884 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.885110    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 12:56:22.888884 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.885113    2573 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 12:56:22.888884 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.885115    2573 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 12:56:22.888884 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.885118    2573 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 12:56:22.888884 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.885121    2573 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 12:56:22.888884 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.885124    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 12:56:22.888884 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.885126    2573 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 12:56:22.888884 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.885129    2573 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 12:56:22.889330 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.885132    2573 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 12:56:22.889330 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.885136    2573 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 12:56:22.889330 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.885138    2573 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 12:56:22.889330 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.885141    2573 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 12:56:22.889330 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.885143    2573 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 12:56:22.889330 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.885146    2573 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 12:56:22.889330 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.885149    2573 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 12:56:22.889330 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.885151    2573 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 12:56:22.889330 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.885154    2573 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 12:56:22.889330 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.885157    2573 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 12:56:22.889330 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.885159    2573 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 12:56:22.889330 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.885162    2573 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 12:56:22.889330 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.885165    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 12:56:22.889330 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.885167    2573 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 12:56:22.889330 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.885170    2573 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 12:56:22.889330 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.885172    2573 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 12:56:22.889330 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.885175    2573 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 12:56:22.889330 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.885177    2573 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 12:56:22.889330 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.885180    2573 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 12:56:22.889330 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.885183    2573 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 12:56:22.889811 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.885185    2573 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 12:56:22.889811 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.885188    2573 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 12:56:22.889811 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.885191    2573 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 12:56:22.889811 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.885194    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 12:56:22.889811 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.885196    2573 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 12:56:22.889811 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.885199    2573 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 12:56:22.889811 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.885201    2573 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 12:56:22.889811 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.885204    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 12:56:22.889811 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.885207    2573 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 12:56:22.889811 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.885210    2573 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 12:56:22.889811 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.885212    2573 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 12:56:22.889811 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.885215    2573 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 12:56:22.889811 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.885218    2573 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 12:56:22.889811 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.885222    2573 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 12:56:22.889811 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.885227    2573 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 12:56:22.889811 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.885230    2573 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 12:56:22.889811 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.885233    2573 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 12:56:22.889811 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.885235    2573 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 12:56:22.889811 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.885238    2573 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 12:56:22.889811 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.885241    2573 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 12:56:22.890328 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.885243    2573 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 12:56:22.890328 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.885246    2573 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 12:56:22.890328 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.885248    2573 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 12:56:22.890328 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.885251    2573 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 12:56:22.890328 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.885253    2573 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 12:56:22.890328 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.885256    2573 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 12:56:22.890328 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.885271    2573 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 12:56:22.890328 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.885274    2573 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 12:56:22.890328 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.885278    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 12:56:22.890328 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.885281    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 12:56:22.890328 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.885283    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 12:56:22.890328 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.885286    2573 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 12:56:22.890328 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.885289    2573 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 12:56:22.890328 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.885292    2573 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 12:56:22.890328 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.885295    2573 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 12:56:22.890328 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.885298    2573 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 12:56:22.890328 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.885301    2573 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 12:56:22.890328 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.885304    2573 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 12:56:22.890328 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.885307    2573 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 12:56:22.890328 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.885309    2573 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 12:56:22.890795 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:22.886074    2573 flags.go:64] FLAG: --address="0.0.0.0"
Oct 30 12:56:22.890795 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:22.886085    2573 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Oct 30 12:56:22.890795 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:22.886091    2573 flags.go:64] FLAG: --anonymous-auth="true"
Oct 30 12:56:22.890795 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:22.886096    2573 flags.go:64] FLAG: --application-metrics-count-limit="100"
Oct 30 12:56:22.890795 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:22.886101    2573 flags.go:64] FLAG: --authentication-token-webhook="false"
Oct 30 12:56:22.890795 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:22.886105    2573 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Oct 30 12:56:22.890795 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:22.886110    2573 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Oct 30 12:56:22.890795 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:22.886114    2573 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Oct 30 12:56:22.890795 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:22.886118    2573 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Oct 30 12:56:22.890795 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:22.886121    2573 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Oct 30 12:56:22.890795 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:22.886125    2573 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Oct 30 12:56:22.890795 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:22.886128    2573 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Oct 30 12:56:22.890795 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:22.886132    2573 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Oct 30 12:56:22.890795 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:22.886135    2573 flags.go:64] FLAG: --cgroup-root=""
Oct 30 12:56:22.890795 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:22.886138    2573 flags.go:64] FLAG: --cgroups-per-qos="true"
Oct 30 12:56:22.890795 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:22.886141    2573 flags.go:64] FLAG: --client-ca-file=""
Oct 30 12:56:22.890795 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:22.886144    2573 flags.go:64] FLAG: --cloud-provider="external"
Oct 30 12:56:22.890795 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:22.886147    2573 flags.go:64] FLAG: --cluster-dns="[]"
Oct 30 12:56:22.890795 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:22.886152    2573 flags.go:64] FLAG: --cluster-domain=""
Oct 30 12:56:22.890795 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:22.886155    2573 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Oct 30 12:56:22.890795 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:22.886159    2573 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Oct 30 12:56:22.890795 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:22.886162    2573 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Oct 30 12:56:22.890795 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:22.886166    2573 flags.go:64] FLAG: --container-log-max-files="5"
Oct 30 12:56:22.890795 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:22.886178    2573 flags.go:64] FLAG: --container-log-max-size="10Mi"
Oct 30 12:56:22.891376 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:22.886184    2573 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Oct 30 12:56:22.891376 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:22.886187    2573 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Oct 30 12:56:22.891376 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:22.886191    2573 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Oct 30 12:56:22.891376 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:22.886194    2573 flags.go:64] FLAG: --contention-profiling="false"
Oct 30 12:56:22.891376 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:22.886198    2573 flags.go:64] FLAG: --cpu-cfs-quota="true"
Oct 30 12:56:22.891376 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:22.886201    2573 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Oct 30 12:56:22.891376 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:22.886204    2573 flags.go:64] FLAG: --cpu-manager-policy="none"
Oct 30 12:56:22.891376 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:22.886207    2573 flags.go:64] FLAG: --cpu-manager-policy-options=""
Oct 30 12:56:22.891376 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:22.886212    2573 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Oct 30 12:56:22.891376 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:22.886215    2573 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Oct 30 12:56:22.891376 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:22.886218    2573 flags.go:64] FLAG: --enable-debugging-handlers="true"
Oct 30 12:56:22.891376 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:22.886222    2573 flags.go:64] FLAG: --enable-load-reader="false"
Oct 30 12:56:22.891376 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:22.886225    2573 flags.go:64] FLAG: --enable-server="true"
Oct 30 12:56:22.891376 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:22.886228    2573 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Oct 30 12:56:22.891376 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:22.886232    2573 flags.go:64] FLAG: --event-burst="100"
Oct 30 12:56:22.891376 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:22.886235    2573 flags.go:64] FLAG: --event-qps="50"
Oct 30 12:56:22.891376 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:22.886239    2573 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Oct 30 12:56:22.891376 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:22.886242    2573 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Oct 30 12:56:22.891376 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:22.886245    2573 flags.go:64] FLAG: --eviction-hard=""
Oct 30 12:56:22.891376 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:22.886249    2573 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Oct 30 12:56:22.891376 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:22.886252    2573 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Oct 30 12:56:22.891376 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:22.886255    2573 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Oct 30 12:56:22.891376 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:22.886270    2573 flags.go:64] FLAG: --eviction-soft=""
Oct 30 12:56:22.891376 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:22.886274    2573 flags.go:64] FLAG: --eviction-soft-grace-period=""
Oct 30 12:56:22.891376 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:22.886277    2573 flags.go:64] FLAG: --exit-on-lock-contention="false"
Oct 30 12:56:22.891987 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:22.886280    2573 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Oct 30 12:56:22.891987 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:22.886283    2573 flags.go:64] FLAG: --experimental-mounter-path=""
Oct 30 12:56:22.891987 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:22.886286    2573 flags.go:64] FLAG: --fail-cgroupv1="true"
Oct 30 12:56:22.891987 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:22.886289    2573 flags.go:64] FLAG: --fail-swap-on="true"
Oct 30 12:56:22.891987 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:22.886292    2573 flags.go:64] FLAG: --feature-gates=""
Oct 30 12:56:22.891987 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:22.886296    2573 flags.go:64] FLAG: --file-check-frequency="20s"
Oct 30 12:56:22.891987 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:22.886299    2573 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Oct 30 12:56:22.891987 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:22.886304    2573 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Oct 30 12:56:22.891987 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:22.886307    2573 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Oct 30 12:56:22.891987 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:22.886311    2573 flags.go:64] FLAG: --healthz-port="10248"
Oct 30 12:56:22.891987 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:22.886314    2573 flags.go:64] FLAG: --help="false"
Oct 30 12:56:22.891987 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:22.886317    2573 flags.go:64] FLAG: --hostname-override="ip-10-0-142-84.ec2.internal"
Oct 30 12:56:22.891987 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:22.886321    2573 flags.go:64] FLAG: --housekeeping-interval="10s"
Oct 30 12:56:22.891987 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:22.886324    2573 flags.go:64] FLAG: --http-check-frequency="20s"
Oct 30 12:56:22.891987 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:22.886327    2573 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Oct 30 12:56:22.891987 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:22.886330    2573 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Oct 30 12:56:22.891987 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:22.886334    2573 flags.go:64] FLAG: --image-gc-high-threshold="85"
Oct 30 12:56:22.891987 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:22.886337    2573 flags.go:64] FLAG: --image-gc-low-threshold="80"
Oct 30 12:56:22.891987 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:22.886341    2573 flags.go:64] FLAG: --image-service-endpoint=""
Oct 30 12:56:22.891987 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:22.886344    2573 flags.go:64] FLAG: --kernel-memcg-notification="false"
Oct 30 12:56:22.891987 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:22.886347    2573 flags.go:64] FLAG: --kube-api-burst="100"
Oct 30 12:56:22.891987 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:22.886351    2573 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Oct 30 12:56:22.891987 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:22.886354    2573 flags.go:64] FLAG: --kube-api-qps="50"
Oct 30 12:56:22.891987 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:22.886357    2573 flags.go:64] FLAG: --kube-reserved=""
Oct 30 12:56:22.892602 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:22.886360    2573 flags.go:64] FLAG: --kube-reserved-cgroup=""
Oct 30 12:56:22.892602 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:22.886363    2573 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Oct 30 12:56:22.892602 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:22.886366    2573 flags.go:64] FLAG: --kubelet-cgroups=""
Oct 30 12:56:22.892602 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:22.886369    2573 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Oct 30 12:56:22.892602 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:22.886371    2573 flags.go:64] FLAG: --lock-file=""
Oct 30 12:56:22.892602 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:22.886374    2573 flags.go:64] FLAG: --log-cadvisor-usage="false"
Oct 30 12:56:22.892602 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:22.886377    2573 flags.go:64] FLAG: --log-flush-frequency="5s"
Oct 30 12:56:22.892602 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:22.886381    2573 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Oct 30 12:56:22.892602 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:22.886387    2573 flags.go:64] FLAG: --log-json-split-stream="false"
Oct 30 12:56:22.892602 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:22.886390    2573 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Oct 30 12:56:22.892602 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:22.886393    2573 flags.go:64] FLAG: --log-text-split-stream="false"
Oct 30 12:56:22.892602 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:22.886396    2573 flags.go:64] FLAG: --logging-format="text"
Oct 30 12:56:22.892602 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:22.886399    2573 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Oct 30 12:56:22.892602 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:22.886402    2573 flags.go:64] FLAG: --make-iptables-util-chains="true"
Oct 30 12:56:22.892602 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:22.886405    2573 flags.go:64] FLAG: --manifest-url=""
Oct 30 12:56:22.892602 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:22.886412    2573 flags.go:64] FLAG: --manifest-url-header=""
Oct 30 12:56:22.892602 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:22.886418    2573 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Oct 30 12:56:22.892602 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:22.886421    2573 flags.go:64] FLAG: --max-open-files="1000000"
Oct 30 12:56:22.892602 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:22.886425    2573 flags.go:64] FLAG: --max-pods="110"
Oct 30 12:56:22.892602 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:22.886428    2573 flags.go:64] FLAG: --maximum-dead-containers="-1"
Oct 30 12:56:22.892602 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:22.886432    2573 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Oct 30 12:56:22.892602 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:22.886435    2573 flags.go:64] FLAG: --memory-manager-policy="None"
Oct 30 12:56:22.892602 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:22.886438    2573 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Oct 30 12:56:22.892602 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:22.886441    2573 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Oct 30 12:56:22.892602 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:22.886444    2573 flags.go:64] FLAG: --node-ip="0.0.0.0"
Oct 30 12:56:22.893368 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:22.886447    2573 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Oct 30 12:56:22.893368 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:22.886456    2573 flags.go:64] FLAG: --node-status-max-images="50"
Oct 30 12:56:22.893368 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:22.886460    2573 flags.go:64] FLAG: --node-status-update-frequency="10s"
Oct 30 12:56:22.893368 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:22.886463    2573 flags.go:64] FLAG: --oom-score-adj="-999"
Oct 30 12:56:22.893368 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:22.886466    2573 flags.go:64] FLAG: --pod-cidr=""
Oct 30 12:56:22.893368 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:22.886469    2573 flags.go:64] FLAG: --pod-manifest-path=""
Oct 30 12:56:22.893368 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:22.886472    2573 flags.go:64] FLAG: --pod-max-pids="-1"
Oct 30 12:56:22.893368 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:22.886475    2573 flags.go:64] FLAG: --pods-per-core="0"
Oct 30 12:56:22.893368 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:22.886478    2573 flags.go:64] FLAG: --port="10250"
Oct 30 12:56:22.893368 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:22.886481    2573 flags.go:64] FLAG: --protect-kernel-defaults="false"
Oct 30 12:56:22.893368 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:22.886484    2573 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0560abecd3e271ce3"
Oct 30 12:56:22.893368 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:22.886488    2573 flags.go:64] FLAG: --qos-reserved=""
Oct 30 12:56:22.893368 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:22.886491    2573 flags.go:64] FLAG: --read-only-port="10255"
Oct 30 12:56:22.893368 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:22.886494    2573 flags.go:64] FLAG: --register-node="true"
Oct 30 12:56:22.893368 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:22.886497    2573 flags.go:64] FLAG: --register-with-taints=""
Oct 30 12:56:22.893368 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:22.886501    2573 flags.go:64] FLAG: --registry-burst="10"
Oct 30 12:56:22.893368 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:22.886504    2573 flags.go:64] FLAG: --registry-qps="5"
Oct 30 12:56:22.893368 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:22.886507    2573 flags.go:64] FLAG: --reserved-cpus=""
Oct 30 12:56:22.893368 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:22.886510    2573 flags.go:64] FLAG: --reserved-memory=""
Oct 30 12:56:22.893368 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:22.886514    2573 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Oct 30 12:56:22.893368 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:22.886517    2573 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Oct 30 12:56:22.893368 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:22.886520    2573 flags.go:64] FLAG: --rotate-certificates="false"
Oct 30 12:56:22.893368 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:22.886525    2573 flags.go:64] FLAG: --rotate-server-certificates="false"
Oct 30 12:56:22.893368 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:22.886528    2573 flags.go:64] FLAG: --runonce="false"
Oct 30 12:56:22.893368 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:22.886531    2573 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Oct 30 12:56:22.893368 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:22.886534    2573 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Oct 30 12:56:22.894510 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:22.886538    2573 flags.go:64] FLAG: --seccomp-default="false"
Oct 30 12:56:22.894510 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:22.886540    2573 flags.go:64] FLAG: --serialize-image-pulls="true"
Oct 30 12:56:22.894510 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:22.886543    2573 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Oct 30 12:56:22.894510 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:22.886547    2573 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Oct 30 12:56:22.894510 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:22.886550    2573 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Oct 30 12:56:22.894510 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:22.886554    2573 flags.go:64] FLAG: --storage-driver-password="root"
Oct 30 12:56:22.894510 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:22.886556    2573 flags.go:64] FLAG: --storage-driver-secure="false"
Oct 30 12:56:22.894510 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:22.886561    2573 flags.go:64] FLAG: --storage-driver-table="stats"
Oct 30 12:56:22.894510 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:22.886564    2573 flags.go:64] FLAG: --storage-driver-user="root"
Oct 30 12:56:22.894510 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:22.886567    2573 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Oct 30 12:56:22.894510 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:22.886570    2573 flags.go:64] FLAG: --sync-frequency="1m0s"
Oct 30 12:56:22.894510 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:22.886573    2573 flags.go:64] FLAG: --system-cgroups=""
Oct 30 12:56:22.894510 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:22.886576    2573 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Oct 30 12:56:22.894510 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:22.886582    2573 flags.go:64] FLAG: --system-reserved-cgroup=""
Oct 30 12:56:22.894510 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:22.886585    2573 flags.go:64] FLAG: --tls-cert-file=""
Oct 30 12:56:22.894510 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:22.886588    2573 flags.go:64] FLAG: --tls-cipher-suites="[]"
Oct 30 12:56:22.894510 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:22.886593    2573 flags.go:64] FLAG: --tls-min-version=""
Oct 30 12:56:22.894510 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:22.886596    2573 flags.go:64] FLAG: --tls-private-key-file=""
Oct 30 12:56:22.894510 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:22.886599    2573 flags.go:64] FLAG: --topology-manager-policy="none"
Oct 30 12:56:22.894510 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:22.886602    2573 flags.go:64] FLAG: --topology-manager-policy-options=""
Oct 30 12:56:22.894510 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:22.886605    2573 flags.go:64] FLAG: --topology-manager-scope="container"
Oct 30 12:56:22.894510 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:22.886608    2573 flags.go:64] FLAG: --v="2"
Oct 30 12:56:22.894510 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:22.886617    2573 flags.go:64] FLAG: --version="false"
Oct 30 12:56:22.894510 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:22.886621    2573 flags.go:64] FLAG: --vmodule=""
Oct 30 12:56:22.894510 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:22.886625    2573 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Oct 30 12:56:22.895555 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:22.886629    2573 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Oct 30 12:56:22.895555 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.886788    2573 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 12:56:22.895555 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.886792    2573 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 12:56:22.895555 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.886796    2573 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 12:56:22.895555 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.886799    2573 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 12:56:22.895555 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.886802    2573 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 12:56:22.895555 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.886805    2573 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 12:56:22.895555 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.886807    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 12:56:22.895555 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.886810    2573 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 12:56:22.895555 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.886813    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 12:56:22.895555 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.886816    2573 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 12:56:22.895555 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.886825    2573 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 12:56:22.895555 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.886828    2573 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 12:56:22.895555 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.886831    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 12:56:22.895555 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.886836    2573 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 12:56:22.895555 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.886839    2573 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 12:56:22.895555 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.886841    2573 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 12:56:22.895555 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.886844    2573 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 12:56:22.895555 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.886847    2573 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 12:56:22.895555 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.886850    2573 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 12:56:22.896173 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.886852    2573 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 12:56:22.896173 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.886855    2573 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 12:56:22.896173 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.886858    2573 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 12:56:22.896173 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.886860    2573 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 12:56:22.896173 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.886863    2573 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 12:56:22.896173 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.886866    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 12:56:22.896173 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.886869    2573 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 12:56:22.896173 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.886871    2573 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 12:56:22.896173 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.886874    2573 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 12:56:22.896173 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.886876    2573 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 12:56:22.896173 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.886879    2573 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 12:56:22.896173 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.886882    2573 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 12:56:22.896173 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.886884    2573 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 12:56:22.896173 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.886887    2573 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 12:56:22.896173 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.886890    2573 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 12:56:22.896173 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.886894    2573 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 12:56:22.896173 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.886897    2573 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 12:56:22.896173 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.886899    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 12:56:22.896173 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.886902    2573 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 12:56:22.896173 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.886904    2573 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 12:56:22.897084 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.886907    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 12:56:22.897084 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.886910    2573 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 12:56:22.897084 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.886912    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 12:56:22.897084 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.886915    2573 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 12:56:22.897084 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.886918    2573 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 12:56:22.897084 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.886921    2573 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 12:56:22.897084 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.886926    2573 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 12:56:22.897084 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.886928    2573 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 12:56:22.897084 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.886931    2573 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 12:56:22.897084 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.886934    2573 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 12:56:22.897084 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.886937    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 12:56:22.897084 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.886939    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 12:56:22.897084 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.886942    2573 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 12:56:22.897084 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.886945    2573 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 12:56:22.897084 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.886948    2573 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 12:56:22.897084 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.886950    2573 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 12:56:22.897084 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.886953    2573 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 12:56:22.897084 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.886956    2573 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 12:56:22.897084 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.886958    2573 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 12:56:22.897944 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.886961    2573 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 12:56:22.897944 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.886963    2573 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 12:56:22.897944 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.886966    2573 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 12:56:22.897944 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.886969    2573 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 12:56:22.897944 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.886972    2573 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 12:56:22.897944 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.886975    2573 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 12:56:22.897944 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.886977    2573 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 12:56:22.897944 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.886981    2573 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 12:56:22.897944 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.886987    2573 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 12:56:22.897944 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.886990    2573 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 12:56:22.897944 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.886993    2573 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 12:56:22.897944 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.886996    2573 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 12:56:22.897944 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.886998    2573 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 12:56:22.897944 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.887001    2573 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 12:56:22.897944 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.887004    2573 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 12:56:22.897944 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.887006    2573 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 12:56:22.897944 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.887009    2573 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 12:56:22.897944 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.887011    2573 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 12:56:22.897944 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.887015    2573 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 12:56:22.898648 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.887019    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 12:56:22.898648 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.887021    2573 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 12:56:22.898648 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.887024    2573 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 12:56:22.898648 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.887027    2573 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 12:56:22.898648 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.887029    2573 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 12:56:22.898648 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.887032    2573 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 12:56:22.898648 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.887035    2573 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 12:56:22.898648 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.887038    2573 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 12:56:22.898648 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.887041    2573 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 12:56:22.898648 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.887043    2573 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 12:56:22.898648 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.887046    2573 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 12:56:22.898648 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.887049    2573 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 12:56:22.898648 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.887051    2573 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 12:56:22.898648 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.887054    2573 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 12:56:22.898648 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.887057    2573 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 12:56:22.898648 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.887059    2573 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 12:56:22.898648 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.887062    2573 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 12:56:22.898648 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.887065    2573 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 12:56:22.898648 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.887067    2573 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 12:56:22.898648 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.887070    2573 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 12:56:22.899126 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.887074    2573 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 12:56:22.899126 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.887078    2573 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 12:56:22.899126 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.887082    2573 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 12:56:22.899126 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.887085    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 12:56:22.899126 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.887087    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 12:56:22.899126 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.887090    2573 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 12:56:22.899126 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.887093    2573 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 12:56:22.899126 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.887095    2573 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 12:56:22.899126 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.887098    2573 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 12:56:22.899126 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:22.887125    2573 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 30 12:56:22.899126 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.887169    2573 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 12:56:22.899126 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.887174    2573 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 12:56:22.899126 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.887177    2573 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 12:56:22.899126 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.887182    2573 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 12:56:22.899126 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.887184    2573 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 12:56:22.899735 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.887187    2573 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 12:56:22.899735 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.887190    2573 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 12:56:22.899735 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.887192    2573 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 12:56:22.899735 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.887195    2573 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 12:56:22.899735 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.887198    2573 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 12:56:22.899735 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.887201    2573 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 12:56:22.899735 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.887203    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 12:56:22.899735 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.887206    2573 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 12:56:22.899735 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.887208    2573 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 12:56:22.899735 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.887211    2573 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 12:56:22.899735 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.887214    2573 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 12:56:22.899735 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.887216    2573 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 12:56:22.899735 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.887219    2573 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 12:56:22.899735 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.887221    2573 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 12:56:22.899735 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.887224    2573 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 12:56:22.899735 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.887227    2573 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 12:56:22.899735 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.887230    2573 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 12:56:22.899735 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.887232    2573 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 12:56:22.899735 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.887235    2573 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 12:56:22.899735 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.887238    2573 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 12:56:22.900375 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.887241    2573 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 12:56:22.900375 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.887243    2573 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 12:56:22.900375 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.887246    2573 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 12:56:22.900375 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.887248    2573 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 12:56:22.900375 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.887251    2573 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 12:56:22.900375 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.887254    2573 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 12:56:22.900375 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.887270    2573 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 12:56:22.900375 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.887274    2573 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 12:56:22.900375 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.887278    2573 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 12:56:22.900375 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.887281    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 12:56:22.900375 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.887286    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 12:56:22.900375 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.887289    2573 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 12:56:22.900375 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.887292    2573 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 12:56:22.900375 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.887295    2573 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 12:56:22.900375 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.887298    2573 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 12:56:22.900375 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.887301    2573 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 12:56:22.900375 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.887303    2573 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 12:56:22.900375 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.887306    2573 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 12:56:22.900375 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.887309    2573 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 12:56:22.900862 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.887311    2573 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 12:56:22.900862 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.887314    2573 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 12:56:22.900862 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.887317    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 12:56:22.900862 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.887319    2573 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 12:56:22.900862 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.887322    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 12:56:22.900862 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.887325    2573 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 12:56:22.900862 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.887328    2573 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 12:56:22.900862 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.887331    2573 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 12:56:22.900862 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.887334    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 12:56:22.900862 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.887336    2573 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 12:56:22.900862 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.887339    2573 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 12:56:22.900862 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.887342    2573 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 12:56:22.900862 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.887345    2573 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 12:56:22.900862 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.887348    2573 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 12:56:22.900862 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.887350    2573 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 12:56:22.900862 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.887353    2573 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 12:56:22.900862 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.887355    2573 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 12:56:22.900862 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.887358    2573 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 12:56:22.900862 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.887361    2573 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 12:56:22.900862 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.887364    2573 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 12:56:22.901581 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.887366    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 12:56:22.901581 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.887369    2573 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 12:56:22.901581 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.887371    2573 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 12:56:22.901581 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.887376    2573 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 12:56:22.901581 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.887379    2573 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 12:56:22.901581 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.887381    2573 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 12:56:22.901581 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.887384    2573 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 12:56:22.901581 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.887387    2573 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 12:56:22.901581 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.887390    2573 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 12:56:22.901581 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.887392    2573 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 12:56:22.901581 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.887395    2573 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 12:56:22.901581 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.887398    2573 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 12:56:22.901581 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.887400    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 12:56:22.901581 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.887403    2573 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 12:56:22.901581 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.887405    2573 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 12:56:22.901581 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.887408    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 12:56:22.901581 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.887411    2573 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 12:56:22.901581 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.887413    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 12:56:22.901581 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.887416    2573 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 12:56:22.901581 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.887419    2573 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 12:56:22.902252 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.887421    2573 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 12:56:22.902252 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.887424    2573 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 12:56:22.902252 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.887426    2573 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 12:56:22.902252 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.887429    2573 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 12:56:22.902252 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.887432    2573 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 12:56:22.902252 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.887434    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 12:56:22.902252 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.887437    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 12:56:22.902252 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.887439    2573 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 12:56:22.902252 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.887442    2573 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 12:56:22.902252 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.887444    2573 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 12:56:22.902252 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.887447    2573 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 12:56:22.902252 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.887450    2573 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 12:56:22.902252 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.887452    2573 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 12:56:22.902252 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.887455    2573 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 12:56:22.902252 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.887457    2573 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 12:56:22.902252 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.887461    2573 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 12:56:22.902252 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.887464    2573 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 12:56:22.902252 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.887468    2573 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 12:56:22.902252 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.887470    2573 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 12:56:22.902860 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.887473    2573 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 12:56:22.902860 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.887475    2573 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 12:56:22.902860 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.887480    2573 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 12:56:22.902860 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:22.895581    2573 server.go:525] "Kubelet version" kubeletVersion="v1.35.9"
Oct 30 12:56:22.902860 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:22.895600    2573 server.go:527] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Oct 30 12:56:22.902860 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:22.896322    2573 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Oct 30 12:56:22.902860 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:22.896336    2573 watchdog_linux.go:138] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Oct 30 12:56:22.902860 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.896423    2573 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 12:56:22.902860 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.896432    2573 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 12:56:22.902860 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.896437    2573 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 12:56:22.902860 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.896442    2573 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 12:56:22.902860 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.896447    2573 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 12:56:22.902860 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.896452    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 12:56:22.902860 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.896456    2573 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 12:56:22.902860 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.896461    2573 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 12:56:22.902860 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.896465    2573 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 12:56:22.902860 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.896470    2573 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 12:56:22.902860 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.896475    2573 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 12:56:22.902860 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.896479    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 12:56:22.903340 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.896484    2573 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 12:56:22.903340 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.896488    2573 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 12:56:22.903340 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.896493    2573 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 12:56:22.903340 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.896498    2573 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 12:56:22.903340 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.896502    2573 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 12:56:22.903340 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.896506    2573 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 12:56:22.903340 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.896511    2573 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 12:56:22.903340 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.896516    2573 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 12:56:22.903340 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.896520    2573 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 12:56:22.903340 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.896525    2573 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 12:56:22.903340 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.896530    2573 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 12:56:22.903340 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.896534    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 12:56:22.903340 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.896538    2573 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 12:56:22.903340 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.896543    2573 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 12:56:22.903340 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.896548    2573 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 12:56:22.903340 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.896570    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 12:56:22.903340 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.896578    2573 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 12:56:22.903340 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.896585    2573 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 12:56:22.903340 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.896591    2573 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 12:56:22.903853 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.896595    2573 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 12:56:22.903853 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.896600    2573 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 12:56:22.903853 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.896605    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 12:56:22.903853 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.896609    2573 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 12:56:22.903853 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.896613    2573 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 12:56:22.903853 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.896618    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 12:56:22.903853 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.896622    2573 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 12:56:22.903853 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.896627    2573 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 12:56:22.903853 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.896631    2573 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 12:56:22.903853 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.896635    2573 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 12:56:22.903853 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.896639    2573 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 12:56:22.903853 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.896643    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 12:56:22.903853 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.896648    2573 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 12:56:22.903853 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.896652    2573 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 12:56:22.903853 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.896657    2573 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 12:56:22.903853 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.896662    2573 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 12:56:22.903853 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.896666    2573 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 12:56:22.903853 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.896671    2573 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 12:56:22.903853 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.896675    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 12:56:22.904378 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.896679    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 12:56:22.904378 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.896683    2573 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 12:56:22.904378 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.896688    2573 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 12:56:22.904378 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.896693    2573 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 12:56:22.904378 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.896697    2573 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 12:56:22.904378 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.896701    2573 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 12:56:22.904378 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.896705    2573 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 12:56:22.904378 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.896710    2573 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 12:56:22.904378 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.896714    2573 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 12:56:22.904378 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.896719    2573 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 12:56:22.904378 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.896723    2573 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 12:56:22.904378 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.896728    2573 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 12:56:22.904378 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.896733    2573 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 12:56:22.904378 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.896738    2573 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 12:56:22.904378 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.896743    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 12:56:22.904378 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.896747    2573 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 12:56:22.904378 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.896751    2573 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 12:56:22.904378 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.896755    2573 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 12:56:22.904378 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.896759    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 12:56:22.904378 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.896764    2573 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 12:56:22.904862 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.896769    2573 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 12:56:22.904862 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.896773    2573 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 12:56:22.904862 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.896777    2573 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 12:56:22.904862 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.896781    2573 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 12:56:22.904862 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.896786    2573 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 12:56:22.904862 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.896792    2573 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 12:56:22.904862 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.896800    2573 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 12:56:22.904862 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.896805    2573 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 12:56:22.904862 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.896810    2573 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 12:56:22.904862 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.896815    2573 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 12:56:22.904862 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.896819    2573 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 12:56:22.904862 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.896823    2573 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 12:56:22.904862 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.896828    2573 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 12:56:22.904862 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.896832    2573 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 12:56:22.904862 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.896836    2573 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 12:56:22.904862 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.896841    2573 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 12:56:22.904862 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.896846    2573 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 12:56:22.904862 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.896850    2573 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 12:56:22.904862 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.896854    2573 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 12:56:22.905527 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.896859    2573 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 12:56:22.905527 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.896863    2573 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 12:56:22.905527 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.896868    2573 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 12:56:22.905527 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.896872    2573 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 12:56:22.905527 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.896877    2573 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 12:56:22.905527 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.896881    2573 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 12:56:22.905527 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.896887    2573 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 12:56:22.905527 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.896892    2573 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 12:56:22.905527 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.896896    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 12:56:22.905527 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.896900    2573 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 12:56:22.905527 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.896905    2573 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 12:56:22.905527 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.896909    2573 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 12:56:22.905527 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.896916    2573 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 12:56:22.905527 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.896921    2573 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 12:56:22.905527 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.896926    2573 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 12:56:22.905527 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.896931    2573 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 12:56:22.905527 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.896935    2573 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 12:56:22.905938 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:22.896978    2573 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 30 12:56:22.905938 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.897197    2573 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 12:56:22.905938 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.897206    2573 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 12:56:22.905938 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.897211    2573 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 12:56:22.905938 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.897216    2573 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 12:56:22.905938 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.897220    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 12:56:22.905938 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.897224    2573 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 12:56:22.905938 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.897229    2573 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 12:56:22.905938 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.897233    2573 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 12:56:22.905938 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.897237    2573 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 12:56:22.905938 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.897241    2573 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 12:56:22.905938 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.897246    2573 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 12:56:22.905938 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.897250    2573 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 12:56:22.905938 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.897255    2573 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 12:56:22.905938 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.897277    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 12:56:22.905938 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.897282    2573 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 12:56:22.906359 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.897286    2573 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 12:56:22.906359 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.897291    2573 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 12:56:22.906359 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.897295    2573 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 12:56:22.906359 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.897300    2573 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 12:56:22.906359 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.897305    2573 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 12:56:22.906359 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.897309    2573 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 12:56:22.906359 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.897318    2573 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 12:56:22.906359 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.897325    2573 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 12:56:22.906359 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.897331    2573 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 12:56:22.906359 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.897336    2573 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 12:56:22.906359 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.897341    2573 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 12:56:22.906359 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.897345    2573 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 12:56:22.906359 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.897351    2573 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 12:56:22.906359 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.897356    2573 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 12:56:22.906359 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.897360    2573 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 12:56:22.906359 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.897365    2573 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 12:56:22.906359 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.897369    2573 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 12:56:22.906359 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.897374    2573 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 12:56:22.906359 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.897378    2573 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 12:56:22.906835 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.897382    2573 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 12:56:22.906835 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.897387    2573 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 12:56:22.906835 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.897391    2573 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 12:56:22.906835 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.897395    2573 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 12:56:22.906835 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.897399    2573 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 12:56:22.906835 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.897404    2573 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 12:56:22.906835 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.897408    2573 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 12:56:22.906835 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.897413    2573 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 12:56:22.906835 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.897417    2573 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 12:56:22.906835 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.897422    2573 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 12:56:22.906835 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.897426    2573 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 12:56:22.906835 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.897430    2573 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 12:56:22.906835 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.897434    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 12:56:22.906835 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.897439    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 12:56:22.906835 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.897444    2573 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 12:56:22.906835 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.897448    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 12:56:22.906835 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.897453    2573 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 12:56:22.906835 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.897457    2573 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 12:56:22.906835 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.897461    2573 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 12:56:22.906835 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.897465    2573 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 12:56:22.907665 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.897470    2573 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 12:56:22.907665 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.897475    2573 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 12:56:22.907665 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.897479    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 12:56:22.907665 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.897484    2573 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 12:56:22.907665 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.897488    2573 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 12:56:22.907665 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.897492    2573 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 12:56:22.907665 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.897497    2573 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 12:56:22.907665 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.897501    2573 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 12:56:22.907665 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.897505    2573 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 12:56:22.907665 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.897509    2573 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 12:56:22.907665 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.897513    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 12:56:22.907665 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.897518    2573 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 12:56:22.907665 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.897522    2573 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 12:56:22.907665 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.897527    2573 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 12:56:22.907665 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.897531    2573 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 12:56:22.907665 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.897535    2573 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 12:56:22.907665 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.897539    2573 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 12:56:22.907665 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.897544    2573 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 12:56:22.907665 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.897549    2573 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 12:56:22.907665 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.897553    2573 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 12:56:22.908418 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.897557    2573 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 12:56:22.908418 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.897562    2573 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 12:56:22.908418 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.897567    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 12:56:22.908418 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.897571    2573 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 12:56:22.908418 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.897576    2573 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 12:56:22.908418 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.897580    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 12:56:22.908418 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.897585    2573 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 12:56:22.908418 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.897589    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 12:56:22.908418 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.897594    2573 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 12:56:22.908418 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.897598    2573 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 12:56:22.908418 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.897602    2573 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 12:56:22.908418 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.897606    2573 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 12:56:22.908418 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.897611    2573 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 12:56:22.908418 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.897616    2573 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 12:56:22.908418 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.897621    2573 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 12:56:22.908418 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.897625    2573 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 12:56:22.908418 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.897629    2573 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 12:56:22.908418 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.897634    2573 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 12:56:22.908418 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.897638    2573 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 12:56:22.908922 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.897642    2573 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 12:56:22.908922 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.897646    2573 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 12:56:22.908922 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.897651    2573 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 12:56:22.908922 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.897655    2573 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 12:56:22.908922 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.897660    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 12:56:22.908922 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.897664    2573 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 12:56:22.908922 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.897671    2573 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 12:56:22.908922 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.897677    2573 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 12:56:22.908922 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.897681    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 12:56:22.908922 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.897685    2573 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 12:56:22.908922 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.897689    2573 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 12:56:22.908922 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.897693    2573 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 12:56:22.908922 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.897698    2573 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 12:56:22.908922 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:22.897739    2573 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 30 12:56:22.908922 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.897790    2573 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 12:56:22.908922 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.897798    2573 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 12:56:22.909342 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.897803    2573 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 12:56:22.909342 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.897808    2573 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 12:56:22.909342 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.897812    2573 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 12:56:22.909342 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.897817    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 12:56:22.909342 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.897821    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 12:56:22.909342 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.897825    2573 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 12:56:22.909342 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.897829    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 12:56:22.909342 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.897833    2573 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 12:56:22.909342 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.897838    2573 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 12:56:22.909342 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.897842    2573 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 12:56:22.909342 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.897846    2573 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 12:56:22.909342 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.897851    2573 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 12:56:22.909342 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.897855    2573 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 12:56:22.909342 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.897859    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 12:56:22.909342 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.897864    2573 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 12:56:22.909342 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.897868    2573 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 12:56:22.909342 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.897873    2573 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 12:56:22.909342 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.897877    2573 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 12:56:22.909342 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.897881    2573 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 12:56:22.909342 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.897885    2573 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 12:56:22.909854 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.897889    2573 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 12:56:22.909854 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.897894    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 12:56:22.909854 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.897898    2573 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 12:56:22.909854 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.897902    2573 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 12:56:22.909854 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.897906    2573 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 12:56:22.909854 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.897910    2573 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 12:56:22.909854 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.897915    2573 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 12:56:22.909854 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.897919    2573 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 12:56:22.909854 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.897924    2573 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 12:56:22.909854 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.897928    2573 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 12:56:22.909854 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.897932    2573 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 12:56:22.909854 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.897936    2573 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 12:56:22.909854 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.897941    2573 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 12:56:22.909854 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.897945    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 12:56:22.909854 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.897949    2573 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 12:56:22.909854 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.897953    2573 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 12:56:22.909854 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.897956    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 12:56:22.909854 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.897961    2573 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 12:56:22.909854 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.897965    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 12:56:22.909854 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.897969    2573 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 12:56:22.910349 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.897973    2573 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 12:56:22.910349 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.897977    2573 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 12:56:22.910349 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.897982    2573 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 12:56:22.910349 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.897987    2573 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 12:56:22.910349 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.897992    2573 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 12:56:22.910349 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.897996    2573 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 12:56:22.910349 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.898001    2573 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 12:56:22.910349 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.898005    2573 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 12:56:22.910349 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.898009    2573 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 12:56:22.910349 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.898013    2573 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 12:56:22.910349 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.898017    2573 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 12:56:22.910349 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.898022    2573 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 12:56:22.910349 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.898026    2573 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 12:56:22.910349 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.898031    2573 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 12:56:22.910349 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.898036    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 12:56:22.910349 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.898040    2573 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 12:56:22.910349 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.898046    2573 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 12:56:22.910349 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.898052    2573 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 12:56:22.910349 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.898056    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 12:56:22.910790 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.898060    2573 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 12:56:22.910790 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.898065    2573 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 12:56:22.910790 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.898069    2573 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 12:56:22.910790 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.898074    2573 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 12:56:22.910790 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.898078    2573 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 12:56:22.910790 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.898083    2573 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 12:56:22.910790 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.898088    2573 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 12:56:22.910790 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.898092    2573 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 12:56:22.910790 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.898096    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 12:56:22.910790 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.898100    2573 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 12:56:22.910790 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.898105    2573 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 12:56:22.910790 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.898109    2573 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 12:56:22.910790 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.898113    2573 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 12:56:22.910790 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.898118    2573 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 12:56:22.910790 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.898123    2573 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 12:56:22.910790 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.898127    2573 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 12:56:22.910790 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.898132    2573 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 12:56:22.910790 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.898136    2573 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 12:56:22.910790 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.898141    2573 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 12:56:22.910790 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.898145    2573 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 12:56:22.911298 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.898149    2573 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 12:56:22.911298 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.898153    2573 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 12:56:22.911298 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.898158    2573 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 12:56:22.911298 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.898162    2573 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 12:56:22.911298 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.898166    2573 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 12:56:22.911298 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.898172    2573 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 12:56:22.911298 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.898178    2573 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 12:56:22.911298 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.898183    2573 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 12:56:22.911298 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.898187    2573 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 12:56:22.911298 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.898192    2573 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 12:56:22.911298 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.898196    2573 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 12:56:22.911298 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.898200    2573 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 12:56:22.911298 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.898205    2573 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 12:56:22.911298 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.898209    2573 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 12:56:22.911298 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.898213    2573 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 12:56:22.911298 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.898217    2573 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 12:56:22.911298 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.898224    2573 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 12:56:22.911298 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.898229    2573 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 12:56:22.911298 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.898233    2573 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 12:56:22.911763 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.898237    2573 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 12:56:22.911763 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.898242    2573 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 12:56:22.911763 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.898246    2573 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 12:56:22.911763 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.898250    2573 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 12:56:22.911763 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.898255    2573 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 12:56:22.911763 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:22.898273    2573 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 12:56:22.911763 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:22.898354    2573 server.go:957] "Client rotation is on, will bootstrap in background"
Oct 30 12:56:22.911763 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:22.902681    2573 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Oct 30 12:56:22.911763 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:22.903606    2573 server.go:1014] "Starting client certificate rotation"
Oct 30 12:56:22.911763 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:22.903699    2573 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Oct 30 12:56:22.911763 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:22.903734    2573 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Oct 30 12:56:22.911763 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:22.909066    2573 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Oct 30 12:56:22.911763 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:22.911655    2573 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Oct 30 12:56:22.922613 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:22.922595    2573 log.go:25] "Validated CRI v1 runtime API"
Oct 30 12:56:22.926873 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:22.926860    2573 log.go:25] "Validated CRI v1 image API"
Oct 30 12:56:22.927713 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:22.927692    2573 server.go:1424] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Oct 30 12:56:22.932440 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:22.932415    2573 fs.go:135] Filesystem UUIDs: map[65651b62-86c7-4011-96cd-b774c13a8db5:/dev/nvme0n1p4 70c3b896-a70f-441a-aee0-27cfdf2fa572:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2]
Oct 30 12:56:22.932501 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:22.932439    2573 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:24 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:26 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:32 fsType:tmpfs blockSize:0} composefs_0-34:{mountpoint:/ major:0 minor:34 fsType:overlay blockSize:0}]
Oct 30 12:56:22.938048 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:22.937937    2573 manager.go:217] Machine: {Timestamp:2026-10-30 12:56:22.936578312 +0000 UTC m=+0.336631664 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3092569 MemoryCapacity:33164337152 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec23c5d664d2c72415e815b707295ddc SystemUUID:ec23c5d6-64d2-c724-15e8-15b707295ddc BootID:a6bea421-76b6-4356-8a08-0ebde2fd07c8 Filesystems:[{Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:24 Capacity:16582168576 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:26 Capacity:6632869888 Type:vfs Inodes:819200 HasInodes:true} {Device:composefs_0-34 DeviceMajor:0 DeviceMinor:34 Capacity:6422528 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:32 Capacity:16582168576 Type:vfs Inodes:1048576 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:ff:f5:4a:d8:11 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:ff:f5:4a:d8:11 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:46:07:dc:bb:70:9f Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164337152 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Oct 30 12:56:22.938118 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:22.938050    2573 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Oct 30 12:56:22.938159 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:22.938127    2573 manager.go:233] Version: {KernelVersion:5.14.0-687.60.1.el9_8.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.8.20261023-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Oct 30 12:56:22.941200 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:22.941179    2573 container_manager_linux.go:275] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Oct 30 12:56:22.941363 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:22.941202    2573 container_manager_linux.go:280] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-142-84.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"/system.slice","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{},"system-reserved-compressible":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Oct 30 12:56:22.941404 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:22.941382    2573 topology_manager.go:143] "Creating topology manager with none policy"
Oct 30 12:56:22.941404 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:22.941390    2573 container_manager_linux.go:311] "Creating device plugin manager"
Oct 30 12:56:22.941493 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:22.941425    2573 manager.go:144] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Oct 30 12:56:22.941493 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:22.941442    2573 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Oct 30 12:56:22.942530 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:22.942520    2573 container_manager_linux.go:320] "Creating Dynamic Resource Allocation (DRA) manager"
Oct 30 12:56:22.946101 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:22.946089    2573 state_mem.go:41] "Initialized" logger="CPUManager state memory"
Oct 30 12:56:22.946217 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:22.946208    2573 server.go:1235] "Using root directory" path="/var/lib/kubelet"
Oct 30 12:56:22.946769 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:22.946759    2573 kubelet.go:493] "Attempting to sync node with API server"
Oct 30 12:56:22.946804 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:22.946777    2573 kubelet.go:394] "Adding static pod path" path="/etc/kubernetes/manifests"
Oct 30 12:56:22.946804 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:22.946788    2573 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Oct 30 12:56:22.946804 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:22.946796    2573 kubelet.go:405] "Adding apiserver pod source"
Oct 30 12:56:22.946804 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:22.946804    2573 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Oct 30 12:56:22.947193 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:22.947178    2573 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Oct 30 12:56:22.949289 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:22.949273    2573 kuberuntime_manager.go:304] "Container runtime initialized" containerRuntime="cri-o" version="1.35.10-3.rhaos4.22.git9383502.el9" apiVersion="v1"
Oct 30 12:56:22.956401 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:22.956389    2573 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Oct 30 12:56:22.957220 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:22.957209    2573 kubelet.go:963] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Oct 30 12:56:22.957275 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:22.957239    2573 kubelet.go:990] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Oct 30 12:56:22.957424 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:22.957412    2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Oct 30 12:56:22.957452 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:22.957430    2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Oct 30 12:56:22.957452 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:22.957436    2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Oct 30 12:56:22.957452 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:22.957442    2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Oct 30 12:56:22.957452 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:22.957447    2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Oct 30 12:56:22.957452 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:22.957453    2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Oct 30 12:56:22.958153 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:22.958145    2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Oct 30 12:56:22.958181 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:22.958154    2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Oct 30 12:56:22.958181 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:22.958161    2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Oct 30 12:56:22.958181 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:22.958167    2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Oct 30 12:56:22.958181 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:22.958175    2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Oct 30 12:56:22.958344 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:22.958184    2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Oct 30 12:56:22.958859 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:22.958850    2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Oct 30 12:56:22.958888 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:22.958863    2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Oct 30 12:56:22.965804 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:22.965789    2573 server.go:1263] "Started kubelet"
Oct 30 12:56:22.965886 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:22.965847    2573 server.go:182] "Starting to listen" address="0.0.0.0" port=10250
Oct 30 12:56:22.965984 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:22.965936    2573 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Oct 30 12:56:22.966034 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:22.965999    2573 server_v1.go:49] "podresources" method="list" useActivePods=true
Oct 30 12:56:22.967102 ip-10-0-142-84 systemd[1]: Started Kubernetes Kubelet.
Oct 30 12:56:22.967494 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:22.967473    2573 server.go:317] "Adding debug handlers to kubelet server"
Oct 30 12:56:22.967556 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:22.967510    2573 server.go:254] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Oct 30 12:56:22.973828 ip-10-0-142-84 kubenswrapper[2573]: E1030 12:56:22.973807    2573 kubelet.go:1679] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Oct 30 12:56:22.974198 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:22.974184    2573 fs_resource_analyzer.go:69] "Starting FS ResourceAnalyzer"
Oct 30 12:56:22.974278 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:22.974203    2573 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Oct 30 12:56:22.974325 ip-10-0-142-84 kubenswrapper[2573]: E1030 12:56:22.974281    2573 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-142-84.ec2.internal\" not found"
Oct 30 12:56:22.974805 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:22.974786    2573 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Oct 30 12:56:22.974805 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:22.974793    2573 volume_manager.go:309] "The desired_state_of_world populator starts"
Oct 30 12:56:22.974948 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:22.974812    2573 volume_manager.go:311] "Starting Kubelet Volume Manager"
Oct 30 12:56:22.974948 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:22.974888    2573 reconstruct.go:97] "Volume reconstruction finished"
Oct 30 12:56:22.974948 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:22.974897    2573 reconciler.go:29] "Reconciler: start to sync state"
Oct 30 12:56:22.975233 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:22.975216    2573 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Oct 30 12:56:22.975233 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:22.975232    2573 factory.go:55] Registering systemd factory
Oct 30 12:56:22.975376 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:22.975240    2573 factory.go:223] Registration of the systemd container factory successfully
Oct 30 12:56:22.975454 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:22.975439    2573 factory.go:153] Registering CRI-O factory
Oct 30 12:56:22.975454 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:22.975453    2573 factory.go:223] Registration of the crio container factory successfully
Oct 30 12:56:22.975547 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:22.975466    2573 factory.go:103] Registering Raw factory
Oct 30 12:56:22.975547 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:22.975485    2573 manager.go:1199] Started watching for new ooms in manager
Oct 30 12:56:22.976366 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:22.976354    2573 manager.go:319] Starting recovery of all containers
Oct 30 12:56:22.986035 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:22.986020    2573 manager.go:324] Recovery completed
Oct 30 12:56:22.991513 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:22.991495    2573 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 12:56:22.993210 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:22.993188    2573 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-84.ec2.internal" event="NodeHasSufficientMemory"
Oct 30 12:56:22.993327 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:22.993228    2573 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-84.ec2.internal" event="NodeHasNoDiskPressure"
Oct 30 12:56:22.993327 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:22.993244    2573 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-84.ec2.internal" event="NodeHasSufficientPID"
Oct 30 12:56:22.993819 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:22.993803    2573 cpu_manager.go:226] "Starting" policy="none"
Oct 30 12:56:22.993819 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:22.993817    2573 cpu_manager.go:227] "Reconciling" reconcilePeriod="10s"
Oct 30 12:56:22.993939 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:22.993837    2573 state_mem.go:41] "Initialized" logger="CPUManager state checkpoint.CPUManager state memory"
Oct 30 12:56:22.993939 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:22.993846    2573 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 12:56:22.994585 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:22.994563    2573 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-84.ec2.internal" event="NodeHasSufficientMemory"
Oct 30 12:56:22.994666 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:22.994597    2573 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-84.ec2.internal" event="NodeHasNoDiskPressure"
Oct 30 12:56:22.994666 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:22.994611    2573 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-84.ec2.internal" event="NodeHasSufficientPID"
Oct 30 12:56:22.995185 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:22.995173    2573 policy_none.go:50] "Start"
Oct 30 12:56:22.995224 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:22.995190    2573 memory_manager.go:187] "Starting memorymanager" policy="None"
Oct 30 12:56:22.995224 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:22.995203    2573 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Oct 30 12:56:22.996439 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:22.996427    2573 policy_none.go:44] "Start"
Oct 30 12:56:23.000528 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:23.000503    2573 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-142-84.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Oct 30 12:56:23.000637 ip-10-0-142-84 kubenswrapper[2573]: E1030 12:56:23.000511    2573 controller.go:201] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-142-84.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Oct 30 12:56:23.001467 ip-10-0-142-84 kubenswrapper[2573]: E1030 12:56:23.000520    2573 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-142-84.ec2.internal.18e34ff34a4c74c3  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-142-84.ec2.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-142-84.ec2.internal,},FirstTimestamp:2026-10-30 12:56:22.965761219 +0000 UTC m=+0.365814569,LastTimestamp:2026-10-30 12:56:22.965761219 +0000 UTC m=+0.365814569,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-142-84.ec2.internal,}"
Oct 30 12:56:23.011576 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:23.011558    2573 reflector.go:446] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Oct 30 12:56:23.016537 ip-10-0-142-84 kubenswrapper[2573]: E1030 12:56:23.016449    2573 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-142-84.ec2.internal.18e34ff34bef4cc6  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-142-84.ec2.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-142-84.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-142-84.ec2.internal,},FirstTimestamp:2026-10-30 12:56:22.993210566 +0000 UTC m=+0.393263918,LastTimestamp:2026-10-30 12:56:22.993210566 +0000 UTC m=+0.393263918,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-142-84.ec2.internal,}"
Oct 30 12:56:23.016641 ip-10-0-142-84 kubenswrapper[2573]: E1030 12:56:23.016589    2573 reflector.go:204] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-142-84.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.Node"
Oct 30 12:56:23.016641 ip-10-0-142-84 kubenswrapper[2573]: E1030 12:56:23.016625    2573 reflector.go:204] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.Service"
Oct 30 12:56:23.016747 ip-10-0-142-84 kubenswrapper[2573]: E1030 12:56:23.016692    2573 reflector.go:204] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.CSIDriver"
Oct 30 12:56:23.021825 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:23.021799    2573 node_container_manager_linux.go:130] "Enforcing system reserved compressible on cgroup" cgroupName="/system.slice" limits={"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"}
Oct 30 12:56:23.024211 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:23.024194    2573 manager.go:341] "Starting Device Plugin manager"
Oct 30 12:56:23.024318 ip-10-0-142-84 kubenswrapper[2573]: E1030 12:56:23.024256    2573 manager.go:525] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Oct 30 12:56:23.024318 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:23.024283    2573 server.go:87] "Starting device plugin registration server"
Oct 30 12:56:23.024603 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:23.024474    2573 eviction_manager.go:194] "Eviction manager: starting control loop"
Oct 30 12:56:23.024603 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:23.024488    2573 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Oct 30 12:56:23.024603 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:23.024555    2573 plugin_watcher.go:53] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Oct 30 12:56:23.024782 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:23.024636    2573 plugin_manager.go:119] "The desired_state_of_world populator (plugin watcher) starts"
Oct 30 12:56:23.024782 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:23.024643    2573 plugin_manager.go:121] "Starting Kubelet Plugin Manager"
Oct 30 12:56:23.025434 ip-10-0-142-84 kubenswrapper[2573]: E1030 12:56:23.025369    2573 eviction_manager.go:272] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Oct 30 12:56:23.025519 ip-10-0-142-84 kubenswrapper[2573]: E1030 12:56:23.025452    2573 eviction_manager.go:297] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-142-84.ec2.internal\" not found"
Oct 30 12:56:23.027337 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:23.027319    2573 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-dhhpd"
Oct 30 12:56:23.032062 ip-10-0-142-84 kubenswrapper[2573]: E1030 12:56:23.032002    2573 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-142-84.ec2.internal.18e34ff34befafcf  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-142-84.ec2.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-142-84.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-142-84.ec2.internal,},FirstTimestamp:2026-10-30 12:56:22.993235919 +0000 UTC m=+0.393289274,LastTimestamp:2026-10-30 12:56:22.993235919 +0000 UTC m=+0.393289274,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-142-84.ec2.internal,}"
Oct 30 12:56:23.050292 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:23.040833    2573 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-dhhpd"
Oct 30 12:56:23.050292 ip-10-0-142-84 kubenswrapper[2573]: E1030 12:56:23.048608    2573 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-142-84.ec2.internal.18e34ff34befe5a4  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-142-84.ec2.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-142-84.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-142-84.ec2.internal,},FirstTimestamp:2026-10-30 12:56:22.9932497 +0000 UTC m=+0.393303051,LastTimestamp:2026-10-30 12:56:22.9932497 +0000 UTC m=+0.393303051,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-142-84.ec2.internal,}"
Oct 30 12:56:23.051715 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:23.051510    2573 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Oct 30 12:56:23.052636 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:23.052617    2573 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Oct 30 12:56:23.052636 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:23.052638    2573 status_manager.go:249] "Starting to sync pod status with apiserver"
Oct 30 12:56:23.052779 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:23.052657    2573 kubelet.go:2521] "Starting kubelet main sync loop"
Oct 30 12:56:23.052779 ip-10-0-142-84 kubenswrapper[2573]: E1030 12:56:23.052691    2573 kubelet.go:2545] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Oct 30 12:56:23.070548 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:23.070532    2573 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 12:56:23.125096 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:23.125074    2573 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 12:56:23.125843 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:23.125827    2573 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-84.ec2.internal" event="NodeHasSufficientMemory"
Oct 30 12:56:23.125917 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:23.125854    2573 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-84.ec2.internal" event="NodeHasNoDiskPressure"
Oct 30 12:56:23.125917 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:23.125872    2573 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-84.ec2.internal" event="NodeHasSufficientPID"
Oct 30 12:56:23.125917 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:23.125896    2573 kubelet_node_status.go:77] "Attempting to register node" node="ip-10-0-142-84.ec2.internal"
Oct 30 12:56:23.145492 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:23.145457    2573 kubelet_node_status.go:80] "Successfully registered node" node="ip-10-0-142-84.ec2.internal"
Oct 30 12:56:23.145492 ip-10-0-142-84 kubenswrapper[2573]: E1030 12:56:23.145479    2573 kubelet_node_status.go:524] "Error updating node status, will retry" err="error getting node \"ip-10-0-142-84.ec2.internal\": node \"ip-10-0-142-84.ec2.internal\" not found"
Oct 30 12:56:23.152920 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:23.152896    2573 kubelet.go:2608] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-84.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-142-84.ec2.internal"]
Oct 30 12:56:23.152986 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:23.152969    2573 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 12:56:23.153719 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:23.153703    2573 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-84.ec2.internal" event="NodeHasSufficientMemory"
Oct 30 12:56:23.153783 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:23.153730    2573 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-84.ec2.internal" event="NodeHasNoDiskPressure"
Oct 30 12:56:23.153783 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:23.153739    2573 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-84.ec2.internal" event="NodeHasSufficientPID"
Oct 30 12:56:23.154948 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:23.154937    2573 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 12:56:23.155083 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:23.155070    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-84.ec2.internal"
Oct 30 12:56:23.155133 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:23.155098    2573 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 12:56:23.155615 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:23.155600    2573 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-84.ec2.internal" event="NodeHasSufficientMemory"
Oct 30 12:56:23.155721 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:23.155629    2573 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-84.ec2.internal" event="NodeHasNoDiskPressure"
Oct 30 12:56:23.155721 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:23.155655    2573 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-84.ec2.internal" event="NodeHasSufficientPID"
Oct 30 12:56:23.155721 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:23.155703    2573 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-84.ec2.internal" event="NodeHasSufficientMemory"
Oct 30 12:56:23.155872 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:23.155725    2573 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-84.ec2.internal" event="NodeHasNoDiskPressure"
Oct 30 12:56:23.155872 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:23.155740    2573 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-84.ec2.internal" event="NodeHasSufficientPID"
Oct 30 12:56:23.156823 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:23.156810    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-84.ec2.internal"
Oct 30 12:56:23.156883 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:23.156838    2573 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 12:56:23.157596 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:23.157581    2573 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-84.ec2.internal" event="NodeHasSufficientMemory"
Oct 30 12:56:23.157676 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:23.157608    2573 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-84.ec2.internal" event="NodeHasNoDiskPressure"
Oct 30 12:56:23.157676 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:23.157621    2573 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-84.ec2.internal" event="NodeHasSufficientPID"
Oct 30 12:56:23.176393 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:23.176236    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6dab6bee2c58785dc24bb59bb1408683-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-84.ec2.internal\" (UID: \"6dab6bee2c58785dc24bb59bb1408683\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-84.ec2.internal"
Oct 30 12:56:23.176485 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:23.176408    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/0c9cf3ec0e3ab50517e6f80b07fb5fb2-config\") pod \"kube-apiserver-proxy-ip-10-0-142-84.ec2.internal\" (UID: \"0c9cf3ec0e3ab50517e6f80b07fb5fb2\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-84.ec2.internal"
Oct 30 12:56:23.176485 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:23.176432    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/6dab6bee2c58785dc24bb59bb1408683-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-84.ec2.internal\" (UID: \"6dab6bee2c58785dc24bb59bb1408683\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-84.ec2.internal"
Oct 30 12:56:23.186771 ip-10-0-142-84 kubenswrapper[2573]: E1030 12:56:23.186751    2573 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-142-84.ec2.internal\" not found" node="ip-10-0-142-84.ec2.internal"
Oct 30 12:56:23.190435 ip-10-0-142-84 kubenswrapper[2573]: E1030 12:56:23.190418    2573 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-142-84.ec2.internal\" not found" node="ip-10-0-142-84.ec2.internal"
Oct 30 12:56:23.190587 ip-10-0-142-84 kubenswrapper[2573]: E1030 12:56:23.190572    2573 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-142-84.ec2.internal\" not found"
Oct 30 12:56:23.276753 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:23.276728    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/6dab6bee2c58785dc24bb59bb1408683-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-84.ec2.internal\" (UID: \"6dab6bee2c58785dc24bb59bb1408683\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-84.ec2.internal"
Oct 30 12:56:23.276859 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:23.276758    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6dab6bee2c58785dc24bb59bb1408683-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-84.ec2.internal\" (UID: \"6dab6bee2c58785dc24bb59bb1408683\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-84.ec2.internal"
Oct 30 12:56:23.276859 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:23.276779    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/0c9cf3ec0e3ab50517e6f80b07fb5fb2-config\") pod \"kube-apiserver-proxy-ip-10-0-142-84.ec2.internal\" (UID: \"0c9cf3ec0e3ab50517e6f80b07fb5fb2\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-84.ec2.internal"
Oct 30 12:56:23.276859 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:23.276840    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/0c9cf3ec0e3ab50517e6f80b07fb5fb2-config\") pod \"kube-apiserver-proxy-ip-10-0-142-84.ec2.internal\" (UID: \"0c9cf3ec0e3ab50517e6f80b07fb5fb2\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-84.ec2.internal"
Oct 30 12:56:23.276972 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:23.276890    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/6dab6bee2c58785dc24bb59bb1408683-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-84.ec2.internal\" (UID: \"6dab6bee2c58785dc24bb59bb1408683\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-84.ec2.internal"
Oct 30 12:56:23.276972 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:23.276916    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6dab6bee2c58785dc24bb59bb1408683-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-84.ec2.internal\" (UID: \"6dab6bee2c58785dc24bb59bb1408683\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-84.ec2.internal"
Oct 30 12:56:23.291027 ip-10-0-142-84 kubenswrapper[2573]: E1030 12:56:23.291007    2573 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-142-84.ec2.internal\" not found"
Oct 30 12:56:23.391730 ip-10-0-142-84 kubenswrapper[2573]: E1030 12:56:23.391706    2573 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-142-84.ec2.internal\" not found"
Oct 30 12:56:23.491423 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:23.491402    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-84.ec2.internal"
Oct 30 12:56:23.492484 ip-10-0-142-84 kubenswrapper[2573]: E1030 12:56:23.492453    2573 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-142-84.ec2.internal\" not found"
Oct 30 12:56:23.494077 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:23.494057    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-84.ec2.internal"
Oct 30 12:56:23.593530 ip-10-0-142-84 kubenswrapper[2573]: E1030 12:56:23.593499    2573 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-142-84.ec2.internal\" not found"
Oct 30 12:56:23.694033 ip-10-0-142-84 kubenswrapper[2573]: E1030 12:56:23.694005    2573 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-142-84.ec2.internal\" not found"
Oct 30 12:56:23.794500 ip-10-0-142-84 kubenswrapper[2573]: E1030 12:56:23.794446    2573 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-142-84.ec2.internal\" not found"
Oct 30 12:56:23.891867 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:23.891835    2573 reflector.go:446] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 12:56:23.894846 ip-10-0-142-84 kubenswrapper[2573]: E1030 12:56:23.894824    2573 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-142-84.ec2.internal\" not found"
Oct 30 12:56:23.904068 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:23.904043    2573 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Oct 30 12:56:23.904215 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:23.904191    2573 reflector.go:578] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:161: Unexpected watch close - watch lasted less than a second and no items received"
Oct 30 12:56:23.904290 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:23.904245    2573 reflector.go:578] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.Service" err="very short watch: k8s.io/client-go/informers/factory.go:161: Unexpected watch close - watch lasted less than a second and no items received"
Oct 30 12:56:23.977624 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:23.977583    2573 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6dab6bee2c58785dc24bb59bb1408683.slice/crio-afc6d32b630af29555dbee82da400929a106c898110d62a6e248dac42fa99dd1 WatchSource:0}: Error finding container afc6d32b630af29555dbee82da400929a106c898110d62a6e248dac42fa99dd1: Status 404 returned error can't find the container with id afc6d32b630af29555dbee82da400929a106c898110d62a6e248dac42fa99dd1
Oct 30 12:56:23.977921 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:23.977899    2573 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0c9cf3ec0e3ab50517e6f80b07fb5fb2.slice/crio-56df6b59efaeea83af00f4daa02105b051a8b360001d76963a79837c1727270e WatchSource:0}: Error finding container 56df6b59efaeea83af00f4daa02105b051a8b360001d76963a79837c1727270e: Status 404 returned error can't find the container with id 56df6b59efaeea83af00f4daa02105b051a8b360001d76963a79837c1727270e
Oct 30 12:56:23.982583 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:23.982559    2573 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 12:56:23.995581 ip-10-0-142-84 kubenswrapper[2573]: E1030 12:56:23.995562    2573 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-142-84.ec2.internal\" not found"
Oct 30 12:56:24.021797 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:24.021778    2573 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 12:56:24.022797 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:24.022780    2573 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-84.ec2.internal" event="NodeHasSufficientMemory"
Oct 30 12:56:24.022866 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:24.022809    2573 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-84.ec2.internal" event="NodeHasNoDiskPressure"
Oct 30 12:56:24.022866 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:24.022821    2573 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-84.ec2.internal" event="NodeHasSufficientPID"
Oct 30 12:56:24.046021 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:24.045956    2573 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-10-29 12:51:23 +0000 UTC" deadline="2028-06-19 15:41:38.938924224 +0000 UTC"
Oct 30 12:56:24.046021 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:24.045987    2573 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14354h45m14.8929421s"
Oct 30 12:56:24.055836 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:24.055787    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-84.ec2.internal" event={"ID":"0c9cf3ec0e3ab50517e6f80b07fb5fb2","Type":"ContainerStarted","Data":"56df6b59efaeea83af00f4daa02105b051a8b360001d76963a79837c1727270e"}
Oct 30 12:56:24.058854 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:24.058830    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-84.ec2.internal" event={"ID":"6dab6bee2c58785dc24bb59bb1408683","Type":"ContainerStarted","Data":"afc6d32b630af29555dbee82da400929a106c898110d62a6e248dac42fa99dd1"}
Oct 30 12:56:24.095772 ip-10-0-142-84 kubenswrapper[2573]: E1030 12:56:24.095746    2573 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-142-84.ec2.internal\" not found"
Oct 30 12:56:24.196832 ip-10-0-142-84 kubenswrapper[2573]: E1030 12:56:24.196806    2573 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-142-84.ec2.internal\" not found"
Oct 30 12:56:24.212186 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:24.212168    2573 reflector.go:446] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 12:56:24.275725 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:24.275704    2573 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-84.ec2.internal"
Oct 30 12:56:24.302980 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:24.302929    2573 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 30 12:56:24.304047 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:24.304033    2573 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-84.ec2.internal"
Oct 30 12:56:24.325507 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:24.325490    2573 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 30 12:56:24.577001 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:24.576923    2573 reflector.go:446] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 12:56:24.816781 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:24.816750    2573 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 12:56:24.948541 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:24.948509    2573 apiserver.go:52] "Watching apiserver"
Oct 30 12:56:24.976296 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:24.976257    2573 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Oct 30 12:56:24.989896 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:24.989859    2573 reflector.go:446] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Oct 30 12:56:24.990287 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:24.990255    2573 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-multus/multus-2l8x5","openshift-multus/network-metrics-daemon-p7tv6","kube-system/global-pull-secret-syncer-rpdcd","openshift-cluster-node-tuning-operator/tuned-8mfkz","openshift-multus/multus-additional-cni-plugins-m4p7x","openshift-network-operator/iptables-alerter-kb9lc","openshift-ovn-kubernetes/ovnkube-node-7rwcq","kube-system/konnectivity-agent-zth9k","kube-system/kube-apiserver-proxy-ip-10-0-142-84.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8bnjr","openshift-image-registry/node-ca-h8q28","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-84.ec2.internal","openshift-network-diagnostics/network-check-target-l5mdf","openshift-dns/node-resolver-4lfbk"]
Oct 30 12:56:24.992188 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:24.992164    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8bnjr"
Oct 30 12:56:24.993841 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:24.993367    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p7tv6"
Oct 30 12:56:24.993841 ip-10-0-142-84 kubenswrapper[2573]: E1030 12:56:24.993455    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p7tv6" podUID="b11d3d97-ad88-4aa3-aaee-e83e3cf12f6d"
Oct 30 12:56:24.994997 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:24.994953    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l5mdf"
Oct 30 12:56:24.995099 ip-10-0-142-84 kubenswrapper[2573]: E1030 12:56:24.995026    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l5mdf" podUID="8d767e44-14df-4ea6-b391-bdf265773535"
Oct 30 12:56:24.996307 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:24.996216    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-2l8x5"
Oct 30 12:56:24.997823 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:24.997805    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-m4p7x"
Oct 30 12:56:24.998934 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:24.998908    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-kb9lc"
Oct 30 12:56:25.001142 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.000598    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-h8q28"
Oct 30 12:56:25.002737 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.002583    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-8mfkz"
Oct 30 12:56:25.004102 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.004081    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-zth9k"
Oct 30 12:56:25.005470 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.005448    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-7rwcq"
Oct 30 12:56:25.006906 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.006881    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rpdcd"
Oct 30 12:56:25.007002 ip-10-0-142-84 kubenswrapper[2573]: E1030 12:56:25.006952    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rpdcd" podUID="f660a256-a12d-4a4a-823d-4b0ff5dcb3da"
Oct 30 12:56:25.008892 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.008867    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-4lfbk"
Oct 30 12:56:25.011556 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.011450    2573 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Oct 30 12:56:25.011674 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.011649    2573 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-tzbll\""
Oct 30 12:56:25.011785 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.011767    2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Oct 30 12:56:25.011953 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.011936    2573 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-node-identity-dockercfg-sp79l\""
Oct 30 12:56:25.014685 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.014666    2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Oct 30 12:56:25.032968 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.032949    2573 reflector.go:446] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Oct 30 12:56:25.033071 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.033008    2573 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-gbhnx"
Oct 30 12:56:25.046203 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.046168    2573 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-gbhnx"
Oct 30 12:56:25.076120 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.076096    2573 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Oct 30 12:56:25.085516 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.085491    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/9a20c40d-f5f4-4d96-85b6-be69a6cd1828-host-run-multus-certs\") pod \"multus-2l8x5\" (UID: \"9a20c40d-f5f4-4d96-85b6-be69a6cd1828\") " pod="openshift-multus/multus-2l8x5"
Oct 30 12:56:25.085617 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.085532    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9962cf52-bd50-4a97-a699-4da03085eb81-cni-binary-copy\") pod \"multus-additional-cni-plugins-m4p7x\" (UID: \"9962cf52-bd50-4a97-a699-4da03085eb81\") " pod="openshift-multus/multus-additional-cni-plugins-m4p7x"
Oct 30 12:56:25.085617 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.085563    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/51d01732-87a4-42b0-a6d5-a3fac689d7f6-konnectivity-ca\") pod \"konnectivity-agent-zth9k\" (UID: \"51d01732-87a4-42b0-a6d5-a3fac689d7f6\") " pod="kube-system/konnectivity-agent-zth9k"
Oct 30 12:56:25.085730 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.085638    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/eca286e8-0b02-42af-9d15-1d8670b7362c-tmp-dir\") pod \"node-resolver-4lfbk\" (UID: \"eca286e8-0b02-42af-9d15-1d8670b7362c\") " pod="openshift-dns/node-resolver-4lfbk"
Oct 30 12:56:25.085799 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.085730    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9a20c40d-f5f4-4d96-85b6-be69a6cd1828-cnibin\") pod \"multus-2l8x5\" (UID: \"9a20c40d-f5f4-4d96-85b6-be69a6cd1828\") " pod="openshift-multus/multus-2l8x5"
Oct 30 12:56:25.085799 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.085761    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9a20c40d-f5f4-4d96-85b6-be69a6cd1828-cni-binary-copy\") pod \"multus-2l8x5\" (UID: \"9a20c40d-f5f4-4d96-85b6-be69a6cd1828\") " pod="openshift-multus/multus-2l8x5"
Oct 30 12:56:25.085799 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.085779    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9a20c40d-f5f4-4d96-85b6-be69a6cd1828-host-var-lib-kubelet\") pod \"multus-2l8x5\" (UID: \"9a20c40d-f5f4-4d96-85b6-be69a6cd1828\") " pod="openshift-multus/multus-2l8x5"
Oct 30 12:56:25.085799 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.085794    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/9a20c40d-f5f4-4d96-85b6-be69a6cd1828-multus-daemon-config\") pod \"multus-2l8x5\" (UID: \"9a20c40d-f5f4-4d96-85b6-be69a6cd1828\") " pod="openshift-multus/multus-2l8x5"
Oct 30 12:56:25.085962 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.085833    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9a20c40d-f5f4-4d96-85b6-be69a6cd1828-etc-kubernetes\") pod \"multus-2l8x5\" (UID: \"9a20c40d-f5f4-4d96-85b6-be69a6cd1828\") " pod="openshift-multus/multus-2l8x5"
Oct 30 12:56:25.085962 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.085881    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/56ff7ccd-573e-4bb2-a2b7-199a0346ed9f-var-lib-kubelet\") pod \"tuned-8mfkz\" (UID: \"56ff7ccd-573e-4bb2-a2b7-199a0346ed9f\") " pod="openshift-cluster-node-tuning-operator/tuned-8mfkz"
Oct 30 12:56:25.085962 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.085911    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/4817ec63-f4d5-4420-9417-148346ff8bb5-log-socket\") pod \"ovnkube-node-7rwcq\" (UID: \"4817ec63-f4d5-4420-9417-148346ff8bb5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7rwcq"
Oct 30 12:56:25.085962 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.085936    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/4817ec63-f4d5-4420-9417-148346ff8bb5-host-cni-netd\") pod \"ovnkube-node-7rwcq\" (UID: \"4817ec63-f4d5-4420-9417-148346ff8bb5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7rwcq"
Oct 30 12:56:25.085962 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.085951    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4817ec63-f4d5-4420-9417-148346ff8bb5-run-openvswitch\") pod \"ovnkube-node-7rwcq\" (UID: \"4817ec63-f4d5-4420-9417-148346ff8bb5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7rwcq"
Oct 30 12:56:25.086160 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.085982    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/4817ec63-f4d5-4420-9417-148346ff8bb5-ovn-node-metrics-cert\") pod \"ovnkube-node-7rwcq\" (UID: \"4817ec63-f4d5-4420-9417-148346ff8bb5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7rwcq"
Oct 30 12:56:25.086160 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.086008    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/f660a256-a12d-4a4a-823d-4b0ff5dcb3da-kubelet-config\") pod \"global-pull-secret-syncer-rpdcd\" (UID: \"f660a256-a12d-4a4a-823d-4b0ff5dcb3da\") " pod="kube-system/global-pull-secret-syncer-rpdcd"
Oct 30 12:56:25.086160 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.086039    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/56ff7ccd-573e-4bb2-a2b7-199a0346ed9f-etc-modprobe-d\") pod \"tuned-8mfkz\" (UID: \"56ff7ccd-573e-4bb2-a2b7-199a0346ed9f\") " pod="openshift-cluster-node-tuning-operator/tuned-8mfkz"
Oct 30 12:56:25.086160 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.086065    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/56ff7ccd-573e-4bb2-a2b7-199a0346ed9f-etc-kubernetes\") pod \"tuned-8mfkz\" (UID: \"56ff7ccd-573e-4bb2-a2b7-199a0346ed9f\") " pod="openshift-cluster-node-tuning-operator/tuned-8mfkz"
Oct 30 12:56:25.086160 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.086117    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/56ff7ccd-573e-4bb2-a2b7-199a0346ed9f-sys\") pod \"tuned-8mfkz\" (UID: \"56ff7ccd-573e-4bb2-a2b7-199a0346ed9f\") " pod="openshift-cluster-node-tuning-operator/tuned-8mfkz"
Oct 30 12:56:25.086160 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.086146    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/56ff7ccd-573e-4bb2-a2b7-199a0346ed9f-host\") pod \"tuned-8mfkz\" (UID: \"56ff7ccd-573e-4bb2-a2b7-199a0346ed9f\") " pod="openshift-cluster-node-tuning-operator/tuned-8mfkz"
Oct 30 12:56:25.086412 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.086174    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/56ff7ccd-573e-4bb2-a2b7-199a0346ed9f-etc-tuned\") pod \"tuned-8mfkz\" (UID: \"56ff7ccd-573e-4bb2-a2b7-199a0346ed9f\") " pod="openshift-cluster-node-tuning-operator/tuned-8mfkz"
Oct 30 12:56:25.086412 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.086240    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8xmnf\" (UniqueName: \"kubernetes.io/projected/16402666-f516-49d1-be0d-f6ab7234e637-kube-api-access-8xmnf\") pod \"node-ca-h8q28\" (UID: \"16402666-f516-49d1-be0d-f6ab7234e637\") " pod="openshift-image-registry/node-ca-h8q28"
Oct 30 12:56:25.086412 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.086314    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/9a20c40d-f5f4-4d96-85b6-be69a6cd1828-hostroot\") pod \"multus-2l8x5\" (UID: \"9a20c40d-f5f4-4d96-85b6-be69a6cd1828\") " pod="openshift-multus/multus-2l8x5"
Oct 30 12:56:25.086412 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.086341    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/56ff7ccd-573e-4bb2-a2b7-199a0346ed9f-etc-sysconfig\") pod \"tuned-8mfkz\" (UID: \"56ff7ccd-573e-4bb2-a2b7-199a0346ed9f\") " pod="openshift-cluster-node-tuning-operator/tuned-8mfkz"
Oct 30 12:56:25.086412 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.086363    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4817ec63-f4d5-4420-9417-148346ff8bb5-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-7rwcq\" (UID: \"4817ec63-f4d5-4420-9417-148346ff8bb5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7rwcq"
Oct 30 12:56:25.086643 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.086413    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9r47j\" (UniqueName: \"kubernetes.io/projected/ac1dc607-3072-4dbc-87e9-db316bf42dff-kube-api-access-9r47j\") pod \"iptables-alerter-kb9lc\" (UID: \"ac1dc607-3072-4dbc-87e9-db316bf42dff\") " pod="openshift-network-operator/iptables-alerter-kb9lc"
Oct 30 12:56:25.086643 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.086466    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9962cf52-bd50-4a97-a699-4da03085eb81-cnibin\") pod \"multus-additional-cni-plugins-m4p7x\" (UID: \"9962cf52-bd50-4a97-a699-4da03085eb81\") " pod="openshift-multus/multus-additional-cni-plugins-m4p7x"
Oct 30 12:56:25.086643 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.086496    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/56ff7ccd-573e-4bb2-a2b7-199a0346ed9f-etc-systemd\") pod \"tuned-8mfkz\" (UID: \"56ff7ccd-573e-4bb2-a2b7-199a0346ed9f\") " pod="openshift-cluster-node-tuning-operator/tuned-8mfkz"
Oct 30 12:56:25.086643 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.086541    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/4817ec63-f4d5-4420-9417-148346ff8bb5-systemd-units\") pod \"ovnkube-node-7rwcq\" (UID: \"4817ec63-f4d5-4420-9417-148346ff8bb5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7rwcq"
Oct 30 12:56:25.086643 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.086593    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b11d3d97-ad88-4aa3-aaee-e83e3cf12f6d-metrics-certs\") pod \"network-metrics-daemon-p7tv6\" (UID: \"b11d3d97-ad88-4aa3-aaee-e83e3cf12f6d\") " pod="openshift-multus/network-metrics-daemon-p7tv6"
Oct 30 12:56:25.086643 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.086631    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vtsfh\" (UniqueName: \"kubernetes.io/projected/b11d3d97-ad88-4aa3-aaee-e83e3cf12f6d-kube-api-access-vtsfh\") pod \"network-metrics-daemon-p7tv6\" (UID: \"b11d3d97-ad88-4aa3-aaee-e83e3cf12f6d\") " pod="openshift-multus/network-metrics-daemon-p7tv6"
Oct 30 12:56:25.086917 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.086665    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/a3ceb095-c75c-43b3-9de5-7dda644eddb1-registration-dir\") pod \"aws-ebs-csi-driver-node-8bnjr\" (UID: \"a3ceb095-c75c-43b3-9de5-7dda644eddb1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8bnjr"
Oct 30 12:56:25.086917 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.086713    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6rhvm\" (UniqueName: \"kubernetes.io/projected/a3ceb095-c75c-43b3-9de5-7dda644eddb1-kube-api-access-6rhvm\") pod \"aws-ebs-csi-driver-node-8bnjr\" (UID: \"a3ceb095-c75c-43b3-9de5-7dda644eddb1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8bnjr"
Oct 30 12:56:25.086917 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.086747    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c6k8q\" (UniqueName: \"kubernetes.io/projected/9a20c40d-f5f4-4d96-85b6-be69a6cd1828-kube-api-access-c6k8q\") pod \"multus-2l8x5\" (UID: \"9a20c40d-f5f4-4d96-85b6-be69a6cd1828\") " pod="openshift-multus/multus-2l8x5"
Oct 30 12:56:25.086917 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.086787    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/4817ec63-f4d5-4420-9417-148346ff8bb5-run-ovn\") pod \"ovnkube-node-7rwcq\" (UID: \"4817ec63-f4d5-4420-9417-148346ff8bb5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7rwcq"
Oct 30 12:56:25.086917 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.086836    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/51d01732-87a4-42b0-a6d5-a3fac689d7f6-agent-certs\") pod \"konnectivity-agent-zth9k\" (UID: \"51d01732-87a4-42b0-a6d5-a3fac689d7f6\") " pod="kube-system/konnectivity-agent-zth9k"
Oct 30 12:56:25.086917 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.086880    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9a20c40d-f5f4-4d96-85b6-be69a6cd1828-system-cni-dir\") pod \"multus-2l8x5\" (UID: \"9a20c40d-f5f4-4d96-85b6-be69a6cd1828\") " pod="openshift-multus/multus-2l8x5"
Oct 30 12:56:25.086917 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.086909    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9962cf52-bd50-4a97-a699-4da03085eb81-system-cni-dir\") pod \"multus-additional-cni-plugins-m4p7x\" (UID: \"9962cf52-bd50-4a97-a699-4da03085eb81\") " pod="openshift-multus/multus-additional-cni-plugins-m4p7x"
Oct 30 12:56:25.087229 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.086957    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/56ff7ccd-573e-4bb2-a2b7-199a0346ed9f-etc-sysctl-d\") pod \"tuned-8mfkz\" (UID: \"56ff7ccd-573e-4bb2-a2b7-199a0346ed9f\") " pod="openshift-cluster-node-tuning-operator/tuned-8mfkz"
Oct 30 12:56:25.087229 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.087005    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/56ff7ccd-573e-4bb2-a2b7-199a0346ed9f-lib-modules\") pod \"tuned-8mfkz\" (UID: \"56ff7ccd-573e-4bb2-a2b7-199a0346ed9f\") " pod="openshift-cluster-node-tuning-operator/tuned-8mfkz"
Oct 30 12:56:25.087229 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.087028    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7rwfx\" (UniqueName: \"kubernetes.io/projected/4817ec63-f4d5-4420-9417-148346ff8bb5-kube-api-access-7rwfx\") pod \"ovnkube-node-7rwcq\" (UID: \"4817ec63-f4d5-4420-9417-148346ff8bb5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7rwcq"
Oct 30 12:56:25.087229 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.087070    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/16402666-f516-49d1-be0d-f6ab7234e637-serviceca\") pod \"node-ca-h8q28\" (UID: \"16402666-f516-49d1-be0d-f6ab7234e637\") " pod="openshift-image-registry/node-ca-h8q28"
Oct 30 12:56:25.087229 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.087090    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/9a20c40d-f5f4-4d96-85b6-be69a6cd1828-multus-socket-dir-parent\") pod \"multus-2l8x5\" (UID: \"9a20c40d-f5f4-4d96-85b6-be69a6cd1828\") " pod="openshift-multus/multus-2l8x5"
Oct 30 12:56:25.087229 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.087119    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/4817ec63-f4d5-4420-9417-148346ff8bb5-run-systemd\") pod \"ovnkube-node-7rwcq\" (UID: \"4817ec63-f4d5-4420-9417-148346ff8bb5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7rwcq"
Oct 30 12:56:25.087229 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.087168    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4817ec63-f4d5-4420-9417-148346ff8bb5-etc-openvswitch\") pod \"ovnkube-node-7rwcq\" (UID: \"4817ec63-f4d5-4420-9417-148346ff8bb5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7rwcq"
Oct 30 12:56:25.087229 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.087196    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9vr7r\" (UniqueName: \"kubernetes.io/projected/8d767e44-14df-4ea6-b391-bdf265773535-kube-api-access-9vr7r\") pod \"network-check-target-l5mdf\" (UID: \"8d767e44-14df-4ea6-b391-bdf265773535\") " pod="openshift-network-diagnostics/network-check-target-l5mdf"
Oct 30 12:56:25.087229 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.087221    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/56ff7ccd-573e-4bb2-a2b7-199a0346ed9f-run\") pod \"tuned-8mfkz\" (UID: \"56ff7ccd-573e-4bb2-a2b7-199a0346ed9f\") " pod="openshift-cluster-node-tuning-operator/tuned-8mfkz"
Oct 30 12:56:25.087662 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.087253    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/eca286e8-0b02-42af-9d15-1d8670b7362c-hosts-file\") pod \"node-resolver-4lfbk\" (UID: \"eca286e8-0b02-42af-9d15-1d8670b7362c\") " pod="openshift-dns/node-resolver-4lfbk"
Oct 30 12:56:25.087662 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.087378    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/a3ceb095-c75c-43b3-9de5-7dda644eddb1-etc-selinux\") pod \"aws-ebs-csi-driver-node-8bnjr\" (UID: \"a3ceb095-c75c-43b3-9de5-7dda644eddb1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8bnjr"
Oct 30 12:56:25.087662 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.087416    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/9962cf52-bd50-4a97-a699-4da03085eb81-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-m4p7x\" (UID: \"9962cf52-bd50-4a97-a699-4da03085eb81\") " pod="openshift-multus/multus-additional-cni-plugins-m4p7x"
Oct 30 12:56:25.087662 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.087462    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qzmnh\" (UniqueName: \"kubernetes.io/projected/9962cf52-bd50-4a97-a699-4da03085eb81-kube-api-access-qzmnh\") pod \"multus-additional-cni-plugins-m4p7x\" (UID: \"9962cf52-bd50-4a97-a699-4da03085eb81\") " pod="openshift-multus/multus-additional-cni-plugins-m4p7x"
Oct 30 12:56:25.087662 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.087497    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/56ff7ccd-573e-4bb2-a2b7-199a0346ed9f-etc-sysctl-conf\") pod \"tuned-8mfkz\" (UID: \"56ff7ccd-573e-4bb2-a2b7-199a0346ed9f\") " pod="openshift-cluster-node-tuning-operator/tuned-8mfkz"
Oct 30 12:56:25.087662 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.087531    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/f660a256-a12d-4a4a-823d-4b0ff5dcb3da-dbus\") pod \"global-pull-secret-syncer-rpdcd\" (UID: \"f660a256-a12d-4a4a-823d-4b0ff5dcb3da\") " pod="kube-system/global-pull-secret-syncer-rpdcd"
Oct 30 12:56:25.087662 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.087553    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/16402666-f516-49d1-be0d-f6ab7234e637-host\") pod \"node-ca-h8q28\" (UID: \"16402666-f516-49d1-be0d-f6ab7234e637\") " pod="openshift-image-registry/node-ca-h8q28"
Oct 30 12:56:25.087662 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.087601    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j2mgz\" (UniqueName: \"kubernetes.io/projected/eca286e8-0b02-42af-9d15-1d8670b7362c-kube-api-access-j2mgz\") pod \"node-resolver-4lfbk\" (UID: \"eca286e8-0b02-42af-9d15-1d8670b7362c\") " pod="openshift-dns/node-resolver-4lfbk"
Oct 30 12:56:25.087662 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.087633    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/a3ceb095-c75c-43b3-9de5-7dda644eddb1-socket-dir\") pod \"aws-ebs-csi-driver-node-8bnjr\" (UID: \"a3ceb095-c75c-43b3-9de5-7dda644eddb1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8bnjr"
Oct 30 12:56:25.088004 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.087672    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/9a20c40d-f5f4-4d96-85b6-be69a6cd1828-host-run-k8s-cni-cncf-io\") pod \"multus-2l8x5\" (UID: \"9a20c40d-f5f4-4d96-85b6-be69a6cd1828\") " pod="openshift-multus/multus-2l8x5"
Oct 30 12:56:25.088004 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.087694    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9962cf52-bd50-4a97-a699-4da03085eb81-tuning-conf-dir\") pod \"multus-additional-cni-plugins-m4p7x\" (UID: \"9962cf52-bd50-4a97-a699-4da03085eb81\") " pod="openshift-multus/multus-additional-cni-plugins-m4p7x"
Oct 30 12:56:25.088004 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.087732    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/9962cf52-bd50-4a97-a699-4da03085eb81-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-m4p7x\" (UID: \"9962cf52-bd50-4a97-a699-4da03085eb81\") " pod="openshift-multus/multus-additional-cni-plugins-m4p7x"
Oct 30 12:56:25.088004 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.087753    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/56ff7ccd-573e-4bb2-a2b7-199a0346ed9f-tmp\") pod \"tuned-8mfkz\" (UID: \"56ff7ccd-573e-4bb2-a2b7-199a0346ed9f\") " pod="openshift-cluster-node-tuning-operator/tuned-8mfkz"
Oct 30 12:56:25.088004 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.087776    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/4817ec63-f4d5-4420-9417-148346ff8bb5-host-kubelet\") pod \"ovnkube-node-7rwcq\" (UID: \"4817ec63-f4d5-4420-9417-148346ff8bb5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7rwcq"
Oct 30 12:56:25.088004 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.087797    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4817ec63-f4d5-4420-9417-148346ff8bb5-var-lib-openvswitch\") pod \"ovnkube-node-7rwcq\" (UID: \"4817ec63-f4d5-4420-9417-148346ff8bb5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7rwcq"
Oct 30 12:56:25.088004 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.087818    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/4817ec63-f4d5-4420-9417-148346ff8bb5-ovnkube-script-lib\") pod \"ovnkube-node-7rwcq\" (UID: \"4817ec63-f4d5-4420-9417-148346ff8bb5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7rwcq"
Oct 30 12:56:25.088004 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.087839    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9a20c40d-f5f4-4d96-85b6-be69a6cd1828-host-var-lib-cni-bin\") pod \"multus-2l8x5\" (UID: \"9a20c40d-f5f4-4d96-85b6-be69a6cd1828\") " pod="openshift-multus/multus-2l8x5"
Oct 30 12:56:25.088004 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.087864    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9a20c40d-f5f4-4d96-85b6-be69a6cd1828-multus-conf-dir\") pod \"multus-2l8x5\" (UID: \"9a20c40d-f5f4-4d96-85b6-be69a6cd1828\") " pod="openshift-multus/multus-2l8x5"
Oct 30 12:56:25.088004 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.087879    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/4817ec63-f4d5-4420-9417-148346ff8bb5-host-slash\") pod \"ovnkube-node-7rwcq\" (UID: \"4817ec63-f4d5-4420-9417-148346ff8bb5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7rwcq"
Oct 30 12:56:25.088004 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.087905    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4817ec63-f4d5-4420-9417-148346ff8bb5-host-cni-bin\") pod \"ovnkube-node-7rwcq\" (UID: \"4817ec63-f4d5-4420-9417-148346ff8bb5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7rwcq"
Oct 30 12:56:25.088004 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.087945    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f660a256-a12d-4a4a-823d-4b0ff5dcb3da-original-pull-secret\") pod \"global-pull-secret-syncer-rpdcd\" (UID: \"f660a256-a12d-4a4a-823d-4b0ff5dcb3da\") " pod="kube-system/global-pull-secret-syncer-rpdcd"
Oct 30 12:56:25.088004 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.087961    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/ac1dc607-3072-4dbc-87e9-db316bf42dff-iptables-alerter-script\") pod \"iptables-alerter-kb9lc\" (UID: \"ac1dc607-3072-4dbc-87e9-db316bf42dff\") " pod="openshift-network-operator/iptables-alerter-kb9lc"
Oct 30 12:56:25.088004 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.087976    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/a3ceb095-c75c-43b3-9de5-7dda644eddb1-device-dir\") pod \"aws-ebs-csi-driver-node-8bnjr\" (UID: \"a3ceb095-c75c-43b3-9de5-7dda644eddb1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8bnjr"
Oct 30 12:56:25.088535 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.088008    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/4817ec63-f4d5-4420-9417-148346ff8bb5-ovnkube-config\") pod \"ovnkube-node-7rwcq\" (UID: \"4817ec63-f4d5-4420-9417-148346ff8bb5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7rwcq"
Oct 30 12:56:25.088535 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.088032    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/4817ec63-f4d5-4420-9417-148346ff8bb5-env-overrides\") pod \"ovnkube-node-7rwcq\" (UID: \"4817ec63-f4d5-4420-9417-148346ff8bb5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7rwcq"
Oct 30 12:56:25.088535 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.088050    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ac1dc607-3072-4dbc-87e9-db316bf42dff-host-slash\") pod \"iptables-alerter-kb9lc\" (UID: \"ac1dc607-3072-4dbc-87e9-db316bf42dff\") " pod="openshift-network-operator/iptables-alerter-kb9lc"
Oct 30 12:56:25.088535 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.088075    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-66dlr\" (UniqueName: \"kubernetes.io/projected/56ff7ccd-573e-4bb2-a2b7-199a0346ed9f-kube-api-access-66dlr\") pod \"tuned-8mfkz\" (UID: \"56ff7ccd-573e-4bb2-a2b7-199a0346ed9f\") " pod="openshift-cluster-node-tuning-operator/tuned-8mfkz"
Oct 30 12:56:25.088535 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.088103    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4817ec63-f4d5-4420-9417-148346ff8bb5-host-run-netns\") pod \"ovnkube-node-7rwcq\" (UID: \"4817ec63-f4d5-4420-9417-148346ff8bb5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7rwcq"
Oct 30 12:56:25.088535 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.088118    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/4817ec63-f4d5-4420-9417-148346ff8bb5-node-log\") pod \"ovnkube-node-7rwcq\" (UID: \"4817ec63-f4d5-4420-9417-148346ff8bb5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7rwcq"
Oct 30 12:56:25.088535 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.088132    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4817ec63-f4d5-4420-9417-148346ff8bb5-host-run-ovn-kubernetes\") pod \"ovnkube-node-7rwcq\" (UID: \"4817ec63-f4d5-4420-9417-148346ff8bb5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7rwcq"
Oct 30 12:56:25.088535 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.088147    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9a20c40d-f5f4-4d96-85b6-be69a6cd1828-multus-cni-dir\") pod \"multus-2l8x5\" (UID: \"9a20c40d-f5f4-4d96-85b6-be69a6cd1828\") " pod="openshift-multus/multus-2l8x5"
Oct 30 12:56:25.088535 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.088253    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/9a20c40d-f5f4-4d96-85b6-be69a6cd1828-host-var-lib-cni-multus\") pod \"multus-2l8x5\" (UID: \"9a20c40d-f5f4-4d96-85b6-be69a6cd1828\") " pod="openshift-multus/multus-2l8x5"
Oct 30 12:56:25.088535 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.088282    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/a3ceb095-c75c-43b3-9de5-7dda644eddb1-kubelet-dir\") pod \"aws-ebs-csi-driver-node-8bnjr\" (UID: \"a3ceb095-c75c-43b3-9de5-7dda644eddb1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8bnjr"
Oct 30 12:56:25.088535 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.088299    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/a3ceb095-c75c-43b3-9de5-7dda644eddb1-sys-fs\") pod \"aws-ebs-csi-driver-node-8bnjr\" (UID: \"a3ceb095-c75c-43b3-9de5-7dda644eddb1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8bnjr"
Oct 30 12:56:25.088535 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.088314    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9a20c40d-f5f4-4d96-85b6-be69a6cd1828-host-run-netns\") pod \"multus-2l8x5\" (UID: \"9a20c40d-f5f4-4d96-85b6-be69a6cd1828\") " pod="openshift-multus/multus-2l8x5"
Oct 30 12:56:25.120758 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.120733    2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Oct 30 12:56:25.120912 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.120804    2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Oct 30 12:56:25.121415 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.121385    2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Oct 30 12:56:25.121541 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.121422    2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Oct 30 12:56:25.121541 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.121488    2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Oct 30 12:56:25.121541 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.121503    2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Oct 30 12:56:25.121711 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.121564    2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Oct 30 12:56:25.121711 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.121583    2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Oct 30 12:56:25.121711 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.121638    2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Oct 30 12:56:25.121711 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.121671    2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Oct 30 12:56:25.121907 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.121732    2573 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Oct 30 12:56:25.121907 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.121808    2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Oct 30 12:56:25.121907 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.121814    2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Oct 30 12:56:25.121907 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.121858    2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Oct 30 12:56:25.121907 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.121901    2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Oct 30 12:56:25.122155 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.121952    2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Oct 30 12:56:25.122155 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.122047    2573 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-25fk6\""
Oct 30 12:56:25.122155 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.122116    2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Oct 30 12:56:25.122302 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.122158    2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Oct 30 12:56:25.122302 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.122295    2573 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-94phb\""
Oct 30 12:56:25.122426 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.122356    2573 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Oct 30 12:56:25.122488 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.122443    2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Oct 30 12:56:25.122544 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.122511    2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Oct 30 12:56:25.122601 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.122549    2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Oct 30 12:56:25.122601 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.122593    2573 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-cxt4s\""
Oct 30 12:56:25.122705 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.122653    2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Oct 30 12:56:25.122752 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.122729    2573 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-dz42j\""
Oct 30 12:56:25.127837 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.127816    2573 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent-dockercfg-wsrzk\""
Oct 30 12:56:25.127960 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.127821    2573 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-7gdbz\""
Oct 30 12:56:25.133598 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.133576    2573 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-xt92t\""
Oct 30 12:56:25.134871 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.134852    2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Oct 30 12:56:25.188877 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.188852    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9962cf52-bd50-4a97-a699-4da03085eb81-system-cni-dir\") pod \"multus-additional-cni-plugins-m4p7x\" (UID: \"9962cf52-bd50-4a97-a699-4da03085eb81\") " pod="openshift-multus/multus-additional-cni-plugins-m4p7x"
Oct 30 12:56:25.189022 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.188889    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/56ff7ccd-573e-4bb2-a2b7-199a0346ed9f-etc-sysctl-d\") pod \"tuned-8mfkz\" (UID: \"56ff7ccd-573e-4bb2-a2b7-199a0346ed9f\") " pod="openshift-cluster-node-tuning-operator/tuned-8mfkz"
Oct 30 12:56:25.189022 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.188917    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/56ff7ccd-573e-4bb2-a2b7-199a0346ed9f-lib-modules\") pod \"tuned-8mfkz\" (UID: \"56ff7ccd-573e-4bb2-a2b7-199a0346ed9f\") " pod="openshift-cluster-node-tuning-operator/tuned-8mfkz"
Oct 30 12:56:25.189022 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.188954    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7rwfx\" (UniqueName: \"kubernetes.io/projected/4817ec63-f4d5-4420-9417-148346ff8bb5-kube-api-access-7rwfx\") pod \"ovnkube-node-7rwcq\" (UID: \"4817ec63-f4d5-4420-9417-148346ff8bb5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7rwcq"
Oct 30 12:56:25.189022 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.188980    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/16402666-f516-49d1-be0d-f6ab7234e637-serviceca\") pod \"node-ca-h8q28\" (UID: \"16402666-f516-49d1-be0d-f6ab7234e637\") " pod="openshift-image-registry/node-ca-h8q28"
Oct 30 12:56:25.189022 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.189006    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/9a20c40d-f5f4-4d96-85b6-be69a6cd1828-multus-socket-dir-parent\") pod \"multus-2l8x5\" (UID: \"9a20c40d-f5f4-4d96-85b6-be69a6cd1828\") " pod="openshift-multus/multus-2l8x5"
Oct 30 12:56:25.189326 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.189021    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/4817ec63-f4d5-4420-9417-148346ff8bb5-run-systemd\") pod \"ovnkube-node-7rwcq\" (UID: \"4817ec63-f4d5-4420-9417-148346ff8bb5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7rwcq"
Oct 30 12:56:25.189326 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.189051    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4817ec63-f4d5-4420-9417-148346ff8bb5-etc-openvswitch\") pod \"ovnkube-node-7rwcq\" (UID: \"4817ec63-f4d5-4420-9417-148346ff8bb5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7rwcq"
Oct 30 12:56:25.189326 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.189070    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9vr7r\" (UniqueName: \"kubernetes.io/projected/8d767e44-14df-4ea6-b391-bdf265773535-kube-api-access-9vr7r\") pod \"network-check-target-l5mdf\" (UID: \"8d767e44-14df-4ea6-b391-bdf265773535\") " pod="openshift-network-diagnostics/network-check-target-l5mdf"
Oct 30 12:56:25.189326 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.189085    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/56ff7ccd-573e-4bb2-a2b7-199a0346ed9f-run\") pod \"tuned-8mfkz\" (UID: \"56ff7ccd-573e-4bb2-a2b7-199a0346ed9f\") " pod="openshift-cluster-node-tuning-operator/tuned-8mfkz"
Oct 30 12:56:25.189326 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.189108    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/eca286e8-0b02-42af-9d15-1d8670b7362c-hosts-file\") pod \"node-resolver-4lfbk\" (UID: \"eca286e8-0b02-42af-9d15-1d8670b7362c\") " pod="openshift-dns/node-resolver-4lfbk"
Oct 30 12:56:25.189326 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.189141    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/a3ceb095-c75c-43b3-9de5-7dda644eddb1-etc-selinux\") pod \"aws-ebs-csi-driver-node-8bnjr\" (UID: \"a3ceb095-c75c-43b3-9de5-7dda644eddb1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8bnjr"
Oct 30 12:56:25.189326 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.189167    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/9962cf52-bd50-4a97-a699-4da03085eb81-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-m4p7x\" (UID: \"9962cf52-bd50-4a97-a699-4da03085eb81\") " pod="openshift-multus/multus-additional-cni-plugins-m4p7x"
Oct 30 12:56:25.189326 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.189192    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qzmnh\" (UniqueName: \"kubernetes.io/projected/9962cf52-bd50-4a97-a699-4da03085eb81-kube-api-access-qzmnh\") pod \"multus-additional-cni-plugins-m4p7x\" (UID: \"9962cf52-bd50-4a97-a699-4da03085eb81\") " pod="openshift-multus/multus-additional-cni-plugins-m4p7x"
Oct 30 12:56:25.189326 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.189215    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/56ff7ccd-573e-4bb2-a2b7-199a0346ed9f-etc-sysctl-conf\") pod \"tuned-8mfkz\" (UID: \"56ff7ccd-573e-4bb2-a2b7-199a0346ed9f\") " pod="openshift-cluster-node-tuning-operator/tuned-8mfkz"
Oct 30 12:56:25.189326 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.189231    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/f660a256-a12d-4a4a-823d-4b0ff5dcb3da-dbus\") pod \"global-pull-secret-syncer-rpdcd\" (UID: \"f660a256-a12d-4a4a-823d-4b0ff5dcb3da\") " pod="kube-system/global-pull-secret-syncer-rpdcd"
Oct 30 12:56:25.189326 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.189245    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/16402666-f516-49d1-be0d-f6ab7234e637-host\") pod \"node-ca-h8q28\" (UID: \"16402666-f516-49d1-be0d-f6ab7234e637\") " pod="openshift-image-registry/node-ca-h8q28"
Oct 30 12:56:25.189326 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.189264    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j2mgz\" (UniqueName: \"kubernetes.io/projected/eca286e8-0b02-42af-9d15-1d8670b7362c-kube-api-access-j2mgz\") pod \"node-resolver-4lfbk\" (UID: \"eca286e8-0b02-42af-9d15-1d8670b7362c\") " pod="openshift-dns/node-resolver-4lfbk"
Oct 30 12:56:25.189326 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.189290    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/a3ceb095-c75c-43b3-9de5-7dda644eddb1-socket-dir\") pod \"aws-ebs-csi-driver-node-8bnjr\" (UID: \"a3ceb095-c75c-43b3-9de5-7dda644eddb1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8bnjr"
Oct 30 12:56:25.189975 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.189328    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/9a20c40d-f5f4-4d96-85b6-be69a6cd1828-host-run-k8s-cni-cncf-io\") pod \"multus-2l8x5\" (UID: \"9a20c40d-f5f4-4d96-85b6-be69a6cd1828\") " pod="openshift-multus/multus-2l8x5"
Oct 30 12:56:25.189975 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.189358    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9962cf52-bd50-4a97-a699-4da03085eb81-tuning-conf-dir\") pod \"multus-additional-cni-plugins-m4p7x\" (UID: \"9962cf52-bd50-4a97-a699-4da03085eb81\") " pod="openshift-multus/multus-additional-cni-plugins-m4p7x"
Oct 30 12:56:25.189975 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.189380    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/9962cf52-bd50-4a97-a699-4da03085eb81-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-m4p7x\" (UID: \"9962cf52-bd50-4a97-a699-4da03085eb81\") " pod="openshift-multus/multus-additional-cni-plugins-m4p7x"
Oct 30 12:56:25.189975 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.189407    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/56ff7ccd-573e-4bb2-a2b7-199a0346ed9f-tmp\") pod \"tuned-8mfkz\" (UID: \"56ff7ccd-573e-4bb2-a2b7-199a0346ed9f\") " pod="openshift-cluster-node-tuning-operator/tuned-8mfkz"
Oct 30 12:56:25.189975 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.189426    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/4817ec63-f4d5-4420-9417-148346ff8bb5-host-kubelet\") pod \"ovnkube-node-7rwcq\" (UID: \"4817ec63-f4d5-4420-9417-148346ff8bb5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7rwcq"
Oct 30 12:56:25.189975 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.189442    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4817ec63-f4d5-4420-9417-148346ff8bb5-var-lib-openvswitch\") pod \"ovnkube-node-7rwcq\" (UID: \"4817ec63-f4d5-4420-9417-148346ff8bb5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7rwcq"
Oct 30 12:56:25.189975 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.189467    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/4817ec63-f4d5-4420-9417-148346ff8bb5-ovnkube-script-lib\") pod \"ovnkube-node-7rwcq\" (UID: \"4817ec63-f4d5-4420-9417-148346ff8bb5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7rwcq"
Oct 30 12:56:25.189975 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.189491    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9a20c40d-f5f4-4d96-85b6-be69a6cd1828-host-var-lib-cni-bin\") pod \"multus-2l8x5\" (UID: \"9a20c40d-f5f4-4d96-85b6-be69a6cd1828\") " pod="openshift-multus/multus-2l8x5"
Oct 30 12:56:25.189975 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.189510    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9a20c40d-f5f4-4d96-85b6-be69a6cd1828-multus-conf-dir\") pod \"multus-2l8x5\" (UID: \"9a20c40d-f5f4-4d96-85b6-be69a6cd1828\") " pod="openshift-multus/multus-2l8x5"
Oct 30 12:56:25.189975 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.189527    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/4817ec63-f4d5-4420-9417-148346ff8bb5-host-slash\") pod \"ovnkube-node-7rwcq\" (UID: \"4817ec63-f4d5-4420-9417-148346ff8bb5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7rwcq"
Oct 30 12:56:25.189975 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.189550    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4817ec63-f4d5-4420-9417-148346ff8bb5-host-cni-bin\") pod \"ovnkube-node-7rwcq\" (UID: \"4817ec63-f4d5-4420-9417-148346ff8bb5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7rwcq"
Oct 30 12:56:25.189975 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.189572    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f660a256-a12d-4a4a-823d-4b0ff5dcb3da-original-pull-secret\") pod \"global-pull-secret-syncer-rpdcd\" (UID: \"f660a256-a12d-4a4a-823d-4b0ff5dcb3da\") " pod="kube-system/global-pull-secret-syncer-rpdcd"
Oct 30 12:56:25.189975 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.189593    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/ac1dc607-3072-4dbc-87e9-db316bf42dff-iptables-alerter-script\") pod \"iptables-alerter-kb9lc\" (UID: \"ac1dc607-3072-4dbc-87e9-db316bf42dff\") " pod="openshift-network-operator/iptables-alerter-kb9lc"
Oct 30 12:56:25.189975 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.189643    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/a3ceb095-c75c-43b3-9de5-7dda644eddb1-device-dir\") pod \"aws-ebs-csi-driver-node-8bnjr\" (UID: \"a3ceb095-c75c-43b3-9de5-7dda644eddb1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8bnjr"
Oct 30 12:56:25.189975 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.189667    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/4817ec63-f4d5-4420-9417-148346ff8bb5-ovnkube-config\") pod \"ovnkube-node-7rwcq\" (UID: \"4817ec63-f4d5-4420-9417-148346ff8bb5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7rwcq"
Oct 30 12:56:25.189975 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.189693    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/4817ec63-f4d5-4420-9417-148346ff8bb5-env-overrides\") pod \"ovnkube-node-7rwcq\" (UID: \"4817ec63-f4d5-4420-9417-148346ff8bb5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7rwcq"
Oct 30 12:56:25.189975 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.189720    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ac1dc607-3072-4dbc-87e9-db316bf42dff-host-slash\") pod \"iptables-alerter-kb9lc\" (UID: \"ac1dc607-3072-4dbc-87e9-db316bf42dff\") " pod="openshift-network-operator/iptables-alerter-kb9lc"
Oct 30 12:56:25.190765 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.189743    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-66dlr\" (UniqueName: \"kubernetes.io/projected/56ff7ccd-573e-4bb2-a2b7-199a0346ed9f-kube-api-access-66dlr\") pod \"tuned-8mfkz\" (UID: \"56ff7ccd-573e-4bb2-a2b7-199a0346ed9f\") " pod="openshift-cluster-node-tuning-operator/tuned-8mfkz"
Oct 30 12:56:25.190765 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.189765    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4817ec63-f4d5-4420-9417-148346ff8bb5-host-run-netns\") pod \"ovnkube-node-7rwcq\" (UID: \"4817ec63-f4d5-4420-9417-148346ff8bb5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7rwcq"
Oct 30 12:56:25.190765 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.189780    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/4817ec63-f4d5-4420-9417-148346ff8bb5-node-log\") pod \"ovnkube-node-7rwcq\" (UID: \"4817ec63-f4d5-4420-9417-148346ff8bb5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7rwcq"
Oct 30 12:56:25.190765 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.189796    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4817ec63-f4d5-4420-9417-148346ff8bb5-host-run-ovn-kubernetes\") pod \"ovnkube-node-7rwcq\" (UID: \"4817ec63-f4d5-4420-9417-148346ff8bb5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7rwcq"
Oct 30 12:56:25.190765 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.189820    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9a20c40d-f5f4-4d96-85b6-be69a6cd1828-multus-cni-dir\") pod \"multus-2l8x5\" (UID: \"9a20c40d-f5f4-4d96-85b6-be69a6cd1828\") " pod="openshift-multus/multus-2l8x5"
Oct 30 12:56:25.190765 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.189844    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/9a20c40d-f5f4-4d96-85b6-be69a6cd1828-host-var-lib-cni-multus\") pod \"multus-2l8x5\" (UID: \"9a20c40d-f5f4-4d96-85b6-be69a6cd1828\") " pod="openshift-multus/multus-2l8x5"
Oct 30 12:56:25.190765 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.189864    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/a3ceb095-c75c-43b3-9de5-7dda644eddb1-kubelet-dir\") pod \"aws-ebs-csi-driver-node-8bnjr\" (UID: \"a3ceb095-c75c-43b3-9de5-7dda644eddb1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8bnjr"
Oct 30 12:56:25.190765 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.189883    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/a3ceb095-c75c-43b3-9de5-7dda644eddb1-sys-fs\") pod \"aws-ebs-csi-driver-node-8bnjr\" (UID: \"a3ceb095-c75c-43b3-9de5-7dda644eddb1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8bnjr"
Oct 30 12:56:25.190765 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.189901    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9a20c40d-f5f4-4d96-85b6-be69a6cd1828-host-run-netns\") pod \"multus-2l8x5\" (UID: \"9a20c40d-f5f4-4d96-85b6-be69a6cd1828\") " pod="openshift-multus/multus-2l8x5"
Oct 30 12:56:25.190765 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.189932    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/9a20c40d-f5f4-4d96-85b6-be69a6cd1828-host-run-multus-certs\") pod \"multus-2l8x5\" (UID: \"9a20c40d-f5f4-4d96-85b6-be69a6cd1828\") " pod="openshift-multus/multus-2l8x5"
Oct 30 12:56:25.190765 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.189949    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9962cf52-bd50-4a97-a699-4da03085eb81-cni-binary-copy\") pod \"multus-additional-cni-plugins-m4p7x\" (UID: \"9962cf52-bd50-4a97-a699-4da03085eb81\") " pod="openshift-multus/multus-additional-cni-plugins-m4p7x"
Oct 30 12:56:25.190765 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.189968    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/51d01732-87a4-42b0-a6d5-a3fac689d7f6-konnectivity-ca\") pod \"konnectivity-agent-zth9k\" (UID: \"51d01732-87a4-42b0-a6d5-a3fac689d7f6\") " pod="kube-system/konnectivity-agent-zth9k"
Oct 30 12:56:25.190765 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.189992    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/eca286e8-0b02-42af-9d15-1d8670b7362c-tmp-dir\") pod \"node-resolver-4lfbk\" (UID: \"eca286e8-0b02-42af-9d15-1d8670b7362c\") " pod="openshift-dns/node-resolver-4lfbk"
Oct 30 12:56:25.190765 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.190018    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9a20c40d-f5f4-4d96-85b6-be69a6cd1828-cnibin\") pod \"multus-2l8x5\" (UID: \"9a20c40d-f5f4-4d96-85b6-be69a6cd1828\") " pod="openshift-multus/multus-2l8x5"
Oct 30 12:56:25.190765 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.190044    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9a20c40d-f5f4-4d96-85b6-be69a6cd1828-cni-binary-copy\") pod \"multus-2l8x5\" (UID: \"9a20c40d-f5f4-4d96-85b6-be69a6cd1828\") " pod="openshift-multus/multus-2l8x5"
Oct 30 12:56:25.190765 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.190063    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9a20c40d-f5f4-4d96-85b6-be69a6cd1828-host-var-lib-kubelet\") pod \"multus-2l8x5\" (UID: \"9a20c40d-f5f4-4d96-85b6-be69a6cd1828\") " pod="openshift-multus/multus-2l8x5"
Oct 30 12:56:25.190765 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.190080    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/9a20c40d-f5f4-4d96-85b6-be69a6cd1828-multus-daemon-config\") pod \"multus-2l8x5\" (UID: \"9a20c40d-f5f4-4d96-85b6-be69a6cd1828\") " pod="openshift-multus/multus-2l8x5"
Oct 30 12:56:25.190765 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.190104    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9a20c40d-f5f4-4d96-85b6-be69a6cd1828-etc-kubernetes\") pod \"multus-2l8x5\" (UID: \"9a20c40d-f5f4-4d96-85b6-be69a6cd1828\") " pod="openshift-multus/multus-2l8x5"
Oct 30 12:56:25.191352 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.190122    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/56ff7ccd-573e-4bb2-a2b7-199a0346ed9f-var-lib-kubelet\") pod \"tuned-8mfkz\" (UID: \"56ff7ccd-573e-4bb2-a2b7-199a0346ed9f\") " pod="openshift-cluster-node-tuning-operator/tuned-8mfkz"
Oct 30 12:56:25.191352 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.190138    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/4817ec63-f4d5-4420-9417-148346ff8bb5-log-socket\") pod \"ovnkube-node-7rwcq\" (UID: \"4817ec63-f4d5-4420-9417-148346ff8bb5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7rwcq"
Oct 30 12:56:25.191352 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.190153    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/4817ec63-f4d5-4420-9417-148346ff8bb5-host-cni-netd\") pod \"ovnkube-node-7rwcq\" (UID: \"4817ec63-f4d5-4420-9417-148346ff8bb5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7rwcq"
Oct 30 12:56:25.191352 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.190174    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4817ec63-f4d5-4420-9417-148346ff8bb5-run-openvswitch\") pod \"ovnkube-node-7rwcq\" (UID: \"4817ec63-f4d5-4420-9417-148346ff8bb5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7rwcq"
Oct 30 12:56:25.191352 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.190201    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/4817ec63-f4d5-4420-9417-148346ff8bb5-ovn-node-metrics-cert\") pod \"ovnkube-node-7rwcq\" (UID: \"4817ec63-f4d5-4420-9417-148346ff8bb5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7rwcq"
Oct 30 12:56:25.191352 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.190229    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/f660a256-a12d-4a4a-823d-4b0ff5dcb3da-kubelet-config\") pod \"global-pull-secret-syncer-rpdcd\" (UID: \"f660a256-a12d-4a4a-823d-4b0ff5dcb3da\") " pod="kube-system/global-pull-secret-syncer-rpdcd"
Oct 30 12:56:25.191352 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.190256    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/56ff7ccd-573e-4bb2-a2b7-199a0346ed9f-etc-modprobe-d\") pod \"tuned-8mfkz\" (UID: \"56ff7ccd-573e-4bb2-a2b7-199a0346ed9f\") " pod="openshift-cluster-node-tuning-operator/tuned-8mfkz"
Oct 30 12:56:25.191352 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.190285    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/56ff7ccd-573e-4bb2-a2b7-199a0346ed9f-etc-kubernetes\") pod \"tuned-8mfkz\" (UID: \"56ff7ccd-573e-4bb2-a2b7-199a0346ed9f\") " pod="openshift-cluster-node-tuning-operator/tuned-8mfkz"
Oct 30 12:56:25.191352 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.190313    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/56ff7ccd-573e-4bb2-a2b7-199a0346ed9f-sys\") pod \"tuned-8mfkz\" (UID: \"56ff7ccd-573e-4bb2-a2b7-199a0346ed9f\") " pod="openshift-cluster-node-tuning-operator/tuned-8mfkz"
Oct 30 12:56:25.191352 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.190335    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/56ff7ccd-573e-4bb2-a2b7-199a0346ed9f-host\") pod \"tuned-8mfkz\" (UID: \"56ff7ccd-573e-4bb2-a2b7-199a0346ed9f\") " pod="openshift-cluster-node-tuning-operator/tuned-8mfkz"
Oct 30 12:56:25.191352 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.190365    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/56ff7ccd-573e-4bb2-a2b7-199a0346ed9f-etc-tuned\") pod \"tuned-8mfkz\" (UID: \"56ff7ccd-573e-4bb2-a2b7-199a0346ed9f\") " pod="openshift-cluster-node-tuning-operator/tuned-8mfkz"
Oct 30 12:56:25.191352 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.190384    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8xmnf\" (UniqueName: \"kubernetes.io/projected/16402666-f516-49d1-be0d-f6ab7234e637-kube-api-access-8xmnf\") pod \"node-ca-h8q28\" (UID: \"16402666-f516-49d1-be0d-f6ab7234e637\") " pod="openshift-image-registry/node-ca-h8q28"
Oct 30 12:56:25.191352 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.190405    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/9a20c40d-f5f4-4d96-85b6-be69a6cd1828-hostroot\") pod \"multus-2l8x5\" (UID: \"9a20c40d-f5f4-4d96-85b6-be69a6cd1828\") " pod="openshift-multus/multus-2l8x5"
Oct 30 12:56:25.191352 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.190443    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/56ff7ccd-573e-4bb2-a2b7-199a0346ed9f-etc-sysconfig\") pod \"tuned-8mfkz\" (UID: \"56ff7ccd-573e-4bb2-a2b7-199a0346ed9f\") " pod="openshift-cluster-node-tuning-operator/tuned-8mfkz"
Oct 30 12:56:25.191352 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.190474    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4817ec63-f4d5-4420-9417-148346ff8bb5-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-7rwcq\" (UID: \"4817ec63-f4d5-4420-9417-148346ff8bb5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7rwcq"
Oct 30 12:56:25.191352 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.190501    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9r47j\" (UniqueName: \"kubernetes.io/projected/ac1dc607-3072-4dbc-87e9-db316bf42dff-kube-api-access-9r47j\") pod \"iptables-alerter-kb9lc\" (UID: \"ac1dc607-3072-4dbc-87e9-db316bf42dff\") " pod="openshift-network-operator/iptables-alerter-kb9lc"
Oct 30 12:56:25.191352 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.190530    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9962cf52-bd50-4a97-a699-4da03085eb81-cnibin\") pod \"multus-additional-cni-plugins-m4p7x\" (UID: \"9962cf52-bd50-4a97-a699-4da03085eb81\") " pod="openshift-multus/multus-additional-cni-plugins-m4p7x"
Oct 30 12:56:25.192201 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.190557    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/56ff7ccd-573e-4bb2-a2b7-199a0346ed9f-etc-systemd\") pod \"tuned-8mfkz\" (UID: \"56ff7ccd-573e-4bb2-a2b7-199a0346ed9f\") " pod="openshift-cluster-node-tuning-operator/tuned-8mfkz"
Oct 30 12:56:25.192201 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.190573    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/4817ec63-f4d5-4420-9417-148346ff8bb5-systemd-units\") pod \"ovnkube-node-7rwcq\" (UID: \"4817ec63-f4d5-4420-9417-148346ff8bb5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7rwcq"
Oct 30 12:56:25.192201 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.190597    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b11d3d97-ad88-4aa3-aaee-e83e3cf12f6d-metrics-certs\") pod \"network-metrics-daemon-p7tv6\" (UID: \"b11d3d97-ad88-4aa3-aaee-e83e3cf12f6d\") " pod="openshift-multus/network-metrics-daemon-p7tv6"
Oct 30 12:56:25.192201 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.190625    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vtsfh\" (UniqueName: \"kubernetes.io/projected/b11d3d97-ad88-4aa3-aaee-e83e3cf12f6d-kube-api-access-vtsfh\") pod \"network-metrics-daemon-p7tv6\" (UID: \"b11d3d97-ad88-4aa3-aaee-e83e3cf12f6d\") " pod="openshift-multus/network-metrics-daemon-p7tv6"
Oct 30 12:56:25.192201 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.190658    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/a3ceb095-c75c-43b3-9de5-7dda644eddb1-registration-dir\") pod \"aws-ebs-csi-driver-node-8bnjr\" (UID: \"a3ceb095-c75c-43b3-9de5-7dda644eddb1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8bnjr"
Oct 30 12:56:25.192201 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.190687    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6rhvm\" (UniqueName: \"kubernetes.io/projected/a3ceb095-c75c-43b3-9de5-7dda644eddb1-kube-api-access-6rhvm\") pod \"aws-ebs-csi-driver-node-8bnjr\" (UID: \"a3ceb095-c75c-43b3-9de5-7dda644eddb1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8bnjr"
Oct 30 12:56:25.192201 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.190720    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c6k8q\" (UniqueName: \"kubernetes.io/projected/9a20c40d-f5f4-4d96-85b6-be69a6cd1828-kube-api-access-c6k8q\") pod \"multus-2l8x5\" (UID: \"9a20c40d-f5f4-4d96-85b6-be69a6cd1828\") " pod="openshift-multus/multus-2l8x5"
Oct 30 12:56:25.192201 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.190740    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/4817ec63-f4d5-4420-9417-148346ff8bb5-run-ovn\") pod \"ovnkube-node-7rwcq\" (UID: \"4817ec63-f4d5-4420-9417-148346ff8bb5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7rwcq"
Oct 30 12:56:25.192201 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.190757    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/51d01732-87a4-42b0-a6d5-a3fac689d7f6-agent-certs\") pod \"konnectivity-agent-zth9k\" (UID: \"51d01732-87a4-42b0-a6d5-a3fac689d7f6\") " pod="kube-system/konnectivity-agent-zth9k"
Oct 30 12:56:25.192201 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.190777    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9a20c40d-f5f4-4d96-85b6-be69a6cd1828-system-cni-dir\") pod \"multus-2l8x5\" (UID: \"9a20c40d-f5f4-4d96-85b6-be69a6cd1828\") " pod="openshift-multus/multus-2l8x5"
Oct 30 12:56:25.192201 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.190886    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9a20c40d-f5f4-4d96-85b6-be69a6cd1828-system-cni-dir\") pod \"multus-2l8x5\" (UID: \"9a20c40d-f5f4-4d96-85b6-be69a6cd1828\") " pod="openshift-multus/multus-2l8x5"
Oct 30 12:56:25.192201 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.190940    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9962cf52-bd50-4a97-a699-4da03085eb81-system-cni-dir\") pod \"multus-additional-cni-plugins-m4p7x\" (UID: \"9962cf52-bd50-4a97-a699-4da03085eb81\") " pod="openshift-multus/multus-additional-cni-plugins-m4p7x"
Oct 30 12:56:25.192201 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.191039    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/56ff7ccd-573e-4bb2-a2b7-199a0346ed9f-etc-sysctl-d\") pod \"tuned-8mfkz\" (UID: \"56ff7ccd-573e-4bb2-a2b7-199a0346ed9f\") " pod="openshift-cluster-node-tuning-operator/tuned-8mfkz"
Oct 30 12:56:25.192201 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.191132    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/56ff7ccd-573e-4bb2-a2b7-199a0346ed9f-lib-modules\") pod \"tuned-8mfkz\" (UID: \"56ff7ccd-573e-4bb2-a2b7-199a0346ed9f\") " pod="openshift-cluster-node-tuning-operator/tuned-8mfkz"
Oct 30 12:56:25.192201 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.191892    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/16402666-f516-49d1-be0d-f6ab7234e637-serviceca\") pod \"node-ca-h8q28\" (UID: \"16402666-f516-49d1-be0d-f6ab7234e637\") " pod="openshift-image-registry/node-ca-h8q28"
Oct 30 12:56:25.192201 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.192027    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/9a20c40d-f5f4-4d96-85b6-be69a6cd1828-multus-socket-dir-parent\") pod \"multus-2l8x5\" (UID: \"9a20c40d-f5f4-4d96-85b6-be69a6cd1828\") " pod="openshift-multus/multus-2l8x5"
Oct 30 12:56:25.192201 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.192072    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/4817ec63-f4d5-4420-9417-148346ff8bb5-run-systemd\") pod \"ovnkube-node-7rwcq\" (UID: \"4817ec63-f4d5-4420-9417-148346ff8bb5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7rwcq"
Oct 30 12:56:25.193062 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.192118    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4817ec63-f4d5-4420-9417-148346ff8bb5-etc-openvswitch\") pod \"ovnkube-node-7rwcq\" (UID: \"4817ec63-f4d5-4420-9417-148346ff8bb5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7rwcq"
Oct 30 12:56:25.193062 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.192386    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/56ff7ccd-573e-4bb2-a2b7-199a0346ed9f-run\") pod \"tuned-8mfkz\" (UID: \"56ff7ccd-573e-4bb2-a2b7-199a0346ed9f\") " pod="openshift-cluster-node-tuning-operator/tuned-8mfkz"
Oct 30 12:56:25.193062 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.192433    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/eca286e8-0b02-42af-9d15-1d8670b7362c-hosts-file\") pod \"node-resolver-4lfbk\" (UID: \"eca286e8-0b02-42af-9d15-1d8670b7362c\") " pod="openshift-dns/node-resolver-4lfbk"
Oct 30 12:56:25.193062 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.192512    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/a3ceb095-c75c-43b3-9de5-7dda644eddb1-etc-selinux\") pod \"aws-ebs-csi-driver-node-8bnjr\" (UID: \"a3ceb095-c75c-43b3-9de5-7dda644eddb1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8bnjr"
Oct 30 12:56:25.193062 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.192976    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/9962cf52-bd50-4a97-a699-4da03085eb81-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-m4p7x\" (UID: \"9962cf52-bd50-4a97-a699-4da03085eb81\") " pod="openshift-multus/multus-additional-cni-plugins-m4p7x"
Oct 30 12:56:25.193329 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.193312    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/56ff7ccd-573e-4bb2-a2b7-199a0346ed9f-etc-sysctl-conf\") pod \"tuned-8mfkz\" (UID: \"56ff7ccd-573e-4bb2-a2b7-199a0346ed9f\") " pod="openshift-cluster-node-tuning-operator/tuned-8mfkz"
Oct 30 12:56:25.193409 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.193390    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/f660a256-a12d-4a4a-823d-4b0ff5dcb3da-dbus\") pod \"global-pull-secret-syncer-rpdcd\" (UID: \"f660a256-a12d-4a4a-823d-4b0ff5dcb3da\") " pod="kube-system/global-pull-secret-syncer-rpdcd"
Oct 30 12:56:25.193475 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.193441    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/16402666-f516-49d1-be0d-f6ab7234e637-host\") pod \"node-ca-h8q28\" (UID: \"16402666-f516-49d1-be0d-f6ab7234e637\") " pod="openshift-image-registry/node-ca-h8q28"
Oct 30 12:56:25.193771 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.193754    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/a3ceb095-c75c-43b3-9de5-7dda644eddb1-socket-dir\") pod \"aws-ebs-csi-driver-node-8bnjr\" (UID: \"a3ceb095-c75c-43b3-9de5-7dda644eddb1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8bnjr"
Oct 30 12:56:25.193859 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.193801    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/9a20c40d-f5f4-4d96-85b6-be69a6cd1828-host-run-k8s-cni-cncf-io\") pod \"multus-2l8x5\" (UID: \"9a20c40d-f5f4-4d96-85b6-be69a6cd1828\") " pod="openshift-multus/multus-2l8x5"
Oct 30 12:56:25.193952 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.193937    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9962cf52-bd50-4a97-a699-4da03085eb81-tuning-conf-dir\") pod \"multus-additional-cni-plugins-m4p7x\" (UID: \"9962cf52-bd50-4a97-a699-4da03085eb81\") " pod="openshift-multus/multus-additional-cni-plugins-m4p7x"
Oct 30 12:56:25.194487 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.194421    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/9962cf52-bd50-4a97-a699-4da03085eb81-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-m4p7x\" (UID: \"9962cf52-bd50-4a97-a699-4da03085eb81\") " pod="openshift-multus/multus-additional-cni-plugins-m4p7x"
Oct 30 12:56:25.194736 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.194713    2573 swap_util.go:78] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Oct 30 12:56:25.195105 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.195081    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9a20c40d-f5f4-4d96-85b6-be69a6cd1828-host-var-lib-kubelet\") pod \"multus-2l8x5\" (UID: \"9a20c40d-f5f4-4d96-85b6-be69a6cd1828\") " pod="openshift-multus/multus-2l8x5"
Oct 30 12:56:25.195188 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.195139    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/4817ec63-f4d5-4420-9417-148346ff8bb5-host-kubelet\") pod \"ovnkube-node-7rwcq\" (UID: \"4817ec63-f4d5-4420-9417-148346ff8bb5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7rwcq"
Oct 30 12:56:25.195188 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.195179    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4817ec63-f4d5-4420-9417-148346ff8bb5-var-lib-openvswitch\") pod \"ovnkube-node-7rwcq\" (UID: \"4817ec63-f4d5-4420-9417-148346ff8bb5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7rwcq"
Oct 30 12:56:25.195807 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.195789    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/4817ec63-f4d5-4420-9417-148346ff8bb5-node-log\") pod \"ovnkube-node-7rwcq\" (UID: \"4817ec63-f4d5-4420-9417-148346ff8bb5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7rwcq"
Oct 30 12:56:25.195900 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.195845    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9a20c40d-f5f4-4d96-85b6-be69a6cd1828-host-var-lib-cni-bin\") pod \"multus-2l8x5\" (UID: \"9a20c40d-f5f4-4d96-85b6-be69a6cd1828\") " pod="openshift-multus/multus-2l8x5"
Oct 30 12:56:25.195900 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.195881    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9a20c40d-f5f4-4d96-85b6-be69a6cd1828-multus-conf-dir\") pod \"multus-2l8x5\" (UID: \"9a20c40d-f5f4-4d96-85b6-be69a6cd1828\") " pod="openshift-multus/multus-2l8x5"
Oct 30 12:56:25.195994 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.195909    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/4817ec63-f4d5-4420-9417-148346ff8bb5-host-slash\") pod \"ovnkube-node-7rwcq\" (UID: \"4817ec63-f4d5-4420-9417-148346ff8bb5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7rwcq"
Oct 30 12:56:25.195994 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.195945    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4817ec63-f4d5-4420-9417-148346ff8bb5-host-cni-bin\") pod \"ovnkube-node-7rwcq\" (UID: \"4817ec63-f4d5-4420-9417-148346ff8bb5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7rwcq"
Oct 30 12:56:25.196095 ip-10-0-142-84 kubenswrapper[2573]: E1030 12:56:25.196027    2573 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 30 12:56:25.196151 ip-10-0-142-84 kubenswrapper[2573]: E1030 12:56:25.196098    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f660a256-a12d-4a4a-823d-4b0ff5dcb3da-original-pull-secret podName:f660a256-a12d-4a4a-823d-4b0ff5dcb3da nodeName:}" failed. No retries permitted until 2026-10-30 12:56:25.696069819 +0000 UTC m=+3.096123177 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f660a256-a12d-4a4a-823d-4b0ff5dcb3da-original-pull-secret") pod "global-pull-secret-syncer-rpdcd" (UID: "f660a256-a12d-4a4a-823d-4b0ff5dcb3da") : object "kube-system"/"original-pull-secret" not registered
Oct 30 12:56:25.199696 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.196238    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ac1dc607-3072-4dbc-87e9-db316bf42dff-host-slash\") pod \"iptables-alerter-kb9lc\" (UID: \"ac1dc607-3072-4dbc-87e9-db316bf42dff\") " pod="openshift-network-operator/iptables-alerter-kb9lc"
Oct 30 12:56:25.199696 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.196341    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/a3ceb095-c75c-43b3-9de5-7dda644eddb1-device-dir\") pod \"aws-ebs-csi-driver-node-8bnjr\" (UID: \"a3ceb095-c75c-43b3-9de5-7dda644eddb1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8bnjr"
Oct 30 12:56:25.199696 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.196398    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4817ec63-f4d5-4420-9417-148346ff8bb5-host-run-netns\") pod \"ovnkube-node-7rwcq\" (UID: \"4817ec63-f4d5-4420-9417-148346ff8bb5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7rwcq"
Oct 30 12:56:25.199696 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.196437    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/4817ec63-f4d5-4420-9417-148346ff8bb5-ovnkube-script-lib\") pod \"ovnkube-node-7rwcq\" (UID: \"4817ec63-f4d5-4420-9417-148346ff8bb5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7rwcq"
Oct 30 12:56:25.199696 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.196500    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4817ec63-f4d5-4420-9417-148346ff8bb5-host-run-ovn-kubernetes\") pod \"ovnkube-node-7rwcq\" (UID: \"4817ec63-f4d5-4420-9417-148346ff8bb5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7rwcq"
Oct 30 12:56:25.199696 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.196571    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9a20c40d-f5f4-4d96-85b6-be69a6cd1828-multus-cni-dir\") pod \"multus-2l8x5\" (UID: \"9a20c40d-f5f4-4d96-85b6-be69a6cd1828\") " pod="openshift-multus/multus-2l8x5"
Oct 30 12:56:25.199696 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.196616    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/9a20c40d-f5f4-4d96-85b6-be69a6cd1828-host-var-lib-cni-multus\") pod \"multus-2l8x5\" (UID: \"9a20c40d-f5f4-4d96-85b6-be69a6cd1828\") " pod="openshift-multus/multus-2l8x5"
Oct 30 12:56:25.199696 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.196668    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/a3ceb095-c75c-43b3-9de5-7dda644eddb1-kubelet-dir\") pod \"aws-ebs-csi-driver-node-8bnjr\" (UID: \"a3ceb095-c75c-43b3-9de5-7dda644eddb1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8bnjr"
Oct 30 12:56:25.199696 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.196721    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/a3ceb095-c75c-43b3-9de5-7dda644eddb1-sys-fs\") pod \"aws-ebs-csi-driver-node-8bnjr\" (UID: \"a3ceb095-c75c-43b3-9de5-7dda644eddb1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8bnjr"
Oct 30 12:56:25.199696 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.196765    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9a20c40d-f5f4-4d96-85b6-be69a6cd1828-host-run-netns\") pod \"multus-2l8x5\" (UID: \"9a20c40d-f5f4-4d96-85b6-be69a6cd1828\") " pod="openshift-multus/multus-2l8x5"
Oct 30 12:56:25.199696 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.196810    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/9a20c40d-f5f4-4d96-85b6-be69a6cd1828-host-run-multus-certs\") pod \"multus-2l8x5\" (UID: \"9a20c40d-f5f4-4d96-85b6-be69a6cd1828\") " pod="openshift-multus/multus-2l8x5"
Oct 30 12:56:25.199696 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.197083    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/9a20c40d-f5f4-4d96-85b6-be69a6cd1828-multus-daemon-config\") pod \"multus-2l8x5\" (UID: \"9a20c40d-f5f4-4d96-85b6-be69a6cd1828\") " pod="openshift-multus/multus-2l8x5"
Oct 30 12:56:25.199696 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.197164    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/f660a256-a12d-4a4a-823d-4b0ff5dcb3da-kubelet-config\") pod \"global-pull-secret-syncer-rpdcd\" (UID: \"f660a256-a12d-4a4a-823d-4b0ff5dcb3da\") " pod="kube-system/global-pull-secret-syncer-rpdcd"
Oct 30 12:56:25.199696 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.197250    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/56ff7ccd-573e-4bb2-a2b7-199a0346ed9f-etc-modprobe-d\") pod \"tuned-8mfkz\" (UID: \"56ff7ccd-573e-4bb2-a2b7-199a0346ed9f\") " pod="openshift-cluster-node-tuning-operator/tuned-8mfkz"
Oct 30 12:56:25.199696 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.197284    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9962cf52-bd50-4a97-a699-4da03085eb81-cni-binary-copy\") pod \"multus-additional-cni-plugins-m4p7x\" (UID: \"9962cf52-bd50-4a97-a699-4da03085eb81\") " pod="openshift-multus/multus-additional-cni-plugins-m4p7x"
Oct 30 12:56:25.199696 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.197306    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/56ff7ccd-573e-4bb2-a2b7-199a0346ed9f-etc-kubernetes\") pod \"tuned-8mfkz\" (UID: \"56ff7ccd-573e-4bb2-a2b7-199a0346ed9f\") " pod="openshift-cluster-node-tuning-operator/tuned-8mfkz"
Oct 30 12:56:25.199696 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.197371    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/ac1dc607-3072-4dbc-87e9-db316bf42dff-iptables-alerter-script\") pod \"iptables-alerter-kb9lc\" (UID: \"ac1dc607-3072-4dbc-87e9-db316bf42dff\") " pod="openshift-network-operator/iptables-alerter-kb9lc"
Oct 30 12:56:25.199696 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.197385    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/56ff7ccd-573e-4bb2-a2b7-199a0346ed9f-host\") pod \"tuned-8mfkz\" (UID: \"56ff7ccd-573e-4bb2-a2b7-199a0346ed9f\") " pod="openshift-cluster-node-tuning-operator/tuned-8mfkz"
Oct 30 12:56:25.200763 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.197600    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/4817ec63-f4d5-4420-9417-148346ff8bb5-systemd-units\") pod \"ovnkube-node-7rwcq\" (UID: \"4817ec63-f4d5-4420-9417-148346ff8bb5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7rwcq"
Oct 30 12:56:25.200763 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.197686    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/9a20c40d-f5f4-4d96-85b6-be69a6cd1828-hostroot\") pod \"multus-2l8x5\" (UID: \"9a20c40d-f5f4-4d96-85b6-be69a6cd1828\") " pod="openshift-multus/multus-2l8x5"
Oct 30 12:56:25.200763 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.197748    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/56ff7ccd-573e-4bb2-a2b7-199a0346ed9f-etc-sysconfig\") pod \"tuned-8mfkz\" (UID: \"56ff7ccd-573e-4bb2-a2b7-199a0346ed9f\") " pod="openshift-cluster-node-tuning-operator/tuned-8mfkz"
Oct 30 12:56:25.200763 ip-10-0-142-84 kubenswrapper[2573]: E1030 12:56:25.197783    2573 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 12:56:25.200763 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.197859    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/51d01732-87a4-42b0-a6d5-a3fac689d7f6-konnectivity-ca\") pod \"konnectivity-agent-zth9k\" (UID: \"51d01732-87a4-42b0-a6d5-a3fac689d7f6\") " pod="kube-system/konnectivity-agent-zth9k"
Oct 30 12:56:25.200763 ip-10-0-142-84 kubenswrapper[2573]: E1030 12:56:25.197863    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b11d3d97-ad88-4aa3-aaee-e83e3cf12f6d-metrics-certs podName:b11d3d97-ad88-4aa3-aaee-e83e3cf12f6d nodeName:}" failed. No retries permitted until 2026-10-30 12:56:25.697843402 +0000 UTC m=+3.097896769 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b11d3d97-ad88-4aa3-aaee-e83e3cf12f6d-metrics-certs") pod "network-metrics-daemon-p7tv6" (UID: "b11d3d97-ad88-4aa3-aaee-e83e3cf12f6d") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 12:56:25.200763 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.197949    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/a3ceb095-c75c-43b3-9de5-7dda644eddb1-registration-dir\") pod \"aws-ebs-csi-driver-node-8bnjr\" (UID: \"a3ceb095-c75c-43b3-9de5-7dda644eddb1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8bnjr"
Oct 30 12:56:25.200763 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.197979    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4817ec63-f4d5-4420-9417-148346ff8bb5-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-7rwcq\" (UID: \"4817ec63-f4d5-4420-9417-148346ff8bb5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7rwcq"
Oct 30 12:56:25.200763 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.198145    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9a20c40d-f5f4-4d96-85b6-be69a6cd1828-cnibin\") pod \"multus-2l8x5\" (UID: \"9a20c40d-f5f4-4d96-85b6-be69a6cd1828\") " pod="openshift-multus/multus-2l8x5"
Oct 30 12:56:25.200763 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.198207    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9962cf52-bd50-4a97-a699-4da03085eb81-cnibin\") pod \"multus-additional-cni-plugins-m4p7x\" (UID: \"9962cf52-bd50-4a97-a699-4da03085eb81\") " pod="openshift-multus/multus-additional-cni-plugins-m4p7x"
Oct 30 12:56:25.200763 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.198213    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/4817ec63-f4d5-4420-9417-148346ff8bb5-log-socket\") pod \"ovnkube-node-7rwcq\" (UID: \"4817ec63-f4d5-4420-9417-148346ff8bb5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7rwcq"
Oct 30 12:56:25.200763 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.198252    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/eca286e8-0b02-42af-9d15-1d8670b7362c-tmp-dir\") pod \"node-resolver-4lfbk\" (UID: \"eca286e8-0b02-42af-9d15-1d8670b7362c\") " pod="openshift-dns/node-resolver-4lfbk"
Oct 30 12:56:25.200763 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.198259    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9a20c40d-f5f4-4d96-85b6-be69a6cd1828-etc-kubernetes\") pod \"multus-2l8x5\" (UID: \"9a20c40d-f5f4-4d96-85b6-be69a6cd1828\") " pod="openshift-multus/multus-2l8x5"
Oct 30 12:56:25.200763 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.198333    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/56ff7ccd-573e-4bb2-a2b7-199a0346ed9f-var-lib-kubelet\") pod \"tuned-8mfkz\" (UID: \"56ff7ccd-573e-4bb2-a2b7-199a0346ed9f\") " pod="openshift-cluster-node-tuning-operator/tuned-8mfkz"
Oct 30 12:56:25.200763 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.198375    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/4817ec63-f4d5-4420-9417-148346ff8bb5-run-ovn\") pod \"ovnkube-node-7rwcq\" (UID: \"4817ec63-f4d5-4420-9417-148346ff8bb5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7rwcq"
Oct 30 12:56:25.200763 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.198430    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/56ff7ccd-573e-4bb2-a2b7-199a0346ed9f-etc-systemd\") pod \"tuned-8mfkz\" (UID: \"56ff7ccd-573e-4bb2-a2b7-199a0346ed9f\") " pod="openshift-cluster-node-tuning-operator/tuned-8mfkz"
Oct 30 12:56:25.200763 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.198478    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/4817ec63-f4d5-4420-9417-148346ff8bb5-host-cni-netd\") pod \"ovnkube-node-7rwcq\" (UID: \"4817ec63-f4d5-4420-9417-148346ff8bb5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7rwcq"
Oct 30 12:56:25.200763 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.198539    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4817ec63-f4d5-4420-9417-148346ff8bb5-run-openvswitch\") pod \"ovnkube-node-7rwcq\" (UID: \"4817ec63-f4d5-4420-9417-148346ff8bb5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7rwcq"
Oct 30 12:56:25.201743 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.198683    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/56ff7ccd-573e-4bb2-a2b7-199a0346ed9f-sys\") pod \"tuned-8mfkz\" (UID: \"56ff7ccd-573e-4bb2-a2b7-199a0346ed9f\") " pod="openshift-cluster-node-tuning-operator/tuned-8mfkz"
Oct 30 12:56:25.201743 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.198774    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9a20c40d-f5f4-4d96-85b6-be69a6cd1828-cni-binary-copy\") pod \"multus-2l8x5\" (UID: \"9a20c40d-f5f4-4d96-85b6-be69a6cd1828\") " pod="openshift-multus/multus-2l8x5"
Oct 30 12:56:25.201743 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.199185    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/56ff7ccd-573e-4bb2-a2b7-199a0346ed9f-tmp\") pod \"tuned-8mfkz\" (UID: \"56ff7ccd-573e-4bb2-a2b7-199a0346ed9f\") " pod="openshift-cluster-node-tuning-operator/tuned-8mfkz"
Oct 30 12:56:25.201743 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.199248    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/4817ec63-f4d5-4420-9417-148346ff8bb5-ovnkube-config\") pod \"ovnkube-node-7rwcq\" (UID: \"4817ec63-f4d5-4420-9417-148346ff8bb5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7rwcq"
Oct 30 12:56:25.201743 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.199613    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/4817ec63-f4d5-4420-9417-148346ff8bb5-env-overrides\") pod \"ovnkube-node-7rwcq\" (UID: \"4817ec63-f4d5-4420-9417-148346ff8bb5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7rwcq"
Oct 30 12:56:25.201743 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.200661    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/4817ec63-f4d5-4420-9417-148346ff8bb5-ovn-node-metrics-cert\") pod \"ovnkube-node-7rwcq\" (UID: \"4817ec63-f4d5-4420-9417-148346ff8bb5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7rwcq"
Oct 30 12:56:25.201743 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.201480    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/56ff7ccd-573e-4bb2-a2b7-199a0346ed9f-etc-tuned\") pod \"tuned-8mfkz\" (UID: \"56ff7ccd-573e-4bb2-a2b7-199a0346ed9f\") " pod="openshift-cluster-node-tuning-operator/tuned-8mfkz"
Oct 30 12:56:25.202575 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.202548    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/51d01732-87a4-42b0-a6d5-a3fac689d7f6-agent-certs\") pod \"konnectivity-agent-zth9k\" (UID: \"51d01732-87a4-42b0-a6d5-a3fac689d7f6\") " pod="kube-system/konnectivity-agent-zth9k"
Oct 30 12:56:25.214284 ip-10-0-142-84 kubenswrapper[2573]: E1030 12:56:25.213589    2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 30 12:56:25.214284 ip-10-0-142-84 kubenswrapper[2573]: E1030 12:56:25.213619    2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 30 12:56:25.214284 ip-10-0-142-84 kubenswrapper[2573]: E1030 12:56:25.213632    2573 projected.go:196] Error preparing data for projected volume kube-api-access-9vr7r for pod openshift-network-diagnostics/network-check-target-l5mdf: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 12:56:25.214284 ip-10-0-142-84 kubenswrapper[2573]: E1030 12:56:25.213781    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8d767e44-14df-4ea6-b391-bdf265773535-kube-api-access-9vr7r podName:8d767e44-14df-4ea6-b391-bdf265773535 nodeName:}" failed. No retries permitted until 2026-10-30 12:56:25.713762886 +0000 UTC m=+3.113816241 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-9vr7r" (UniqueName: "kubernetes.io/projected/8d767e44-14df-4ea6-b391-bdf265773535-kube-api-access-9vr7r") pod "network-check-target-l5mdf" (UID: "8d767e44-14df-4ea6-b391-bdf265773535") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 12:56:25.216541 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.216515    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7rwfx\" (UniqueName: \"kubernetes.io/projected/4817ec63-f4d5-4420-9417-148346ff8bb5-kube-api-access-7rwfx\") pod \"ovnkube-node-7rwcq\" (UID: \"4817ec63-f4d5-4420-9417-148346ff8bb5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7rwcq"
Oct 30 12:56:25.216969 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.216945    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qzmnh\" (UniqueName: \"kubernetes.io/projected/9962cf52-bd50-4a97-a699-4da03085eb81-kube-api-access-qzmnh\") pod \"multus-additional-cni-plugins-m4p7x\" (UID: \"9962cf52-bd50-4a97-a699-4da03085eb81\") " pod="openshift-multus/multus-additional-cni-plugins-m4p7x"
Oct 30 12:56:25.217479 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.217456    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j2mgz\" (UniqueName: \"kubernetes.io/projected/eca286e8-0b02-42af-9d15-1d8670b7362c-kube-api-access-j2mgz\") pod \"node-resolver-4lfbk\" (UID: \"eca286e8-0b02-42af-9d15-1d8670b7362c\") " pod="openshift-dns/node-resolver-4lfbk"
Oct 30 12:56:25.220128 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.220076    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8xmnf\" (UniqueName: \"kubernetes.io/projected/16402666-f516-49d1-be0d-f6ab7234e637-kube-api-access-8xmnf\") pod \"node-ca-h8q28\" (UID: \"16402666-f516-49d1-be0d-f6ab7234e637\") " pod="openshift-image-registry/node-ca-h8q28"
Oct 30 12:56:25.224376 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.224329    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6rhvm\" (UniqueName: \"kubernetes.io/projected/a3ceb095-c75c-43b3-9de5-7dda644eddb1-kube-api-access-6rhvm\") pod \"aws-ebs-csi-driver-node-8bnjr\" (UID: \"a3ceb095-c75c-43b3-9de5-7dda644eddb1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8bnjr"
Oct 30 12:56:25.224457 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.224402    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-66dlr\" (UniqueName: \"kubernetes.io/projected/56ff7ccd-573e-4bb2-a2b7-199a0346ed9f-kube-api-access-66dlr\") pod \"tuned-8mfkz\" (UID: \"56ff7ccd-573e-4bb2-a2b7-199a0346ed9f\") " pod="openshift-cluster-node-tuning-operator/tuned-8mfkz"
Oct 30 12:56:25.224754 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.224706    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9r47j\" (UniqueName: \"kubernetes.io/projected/ac1dc607-3072-4dbc-87e9-db316bf42dff-kube-api-access-9r47j\") pod \"iptables-alerter-kb9lc\" (UID: \"ac1dc607-3072-4dbc-87e9-db316bf42dff\") " pod="openshift-network-operator/iptables-alerter-kb9lc"
Oct 30 12:56:25.225070 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.225026    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vtsfh\" (UniqueName: \"kubernetes.io/projected/b11d3d97-ad88-4aa3-aaee-e83e3cf12f6d-kube-api-access-vtsfh\") pod \"network-metrics-daemon-p7tv6\" (UID: \"b11d3d97-ad88-4aa3-aaee-e83e3cf12f6d\") " pod="openshift-multus/network-metrics-daemon-p7tv6"
Oct 30 12:56:25.227268 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.227245    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c6k8q\" (UniqueName: \"kubernetes.io/projected/9a20c40d-f5f4-4d96-85b6-be69a6cd1828-kube-api-access-c6k8q\") pod \"multus-2l8x5\" (UID: \"9a20c40d-f5f4-4d96-85b6-be69a6cd1828\") " pod="openshift-multus/multus-2l8x5"
Oct 30 12:56:25.310434 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.310188    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8bnjr"
Oct 30 12:56:25.315932 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.315867    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-2l8x5"
Oct 30 12:56:25.322246 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:25.322220    2573 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda3ceb095_c75c_43b3_9de5_7dda644eddb1.slice/crio-842ea7f7aa13c414c50b554ba5482c42c5614c8418f34fd986e33d90be9e34d0 WatchSource:0}: Error finding container 842ea7f7aa13c414c50b554ba5482c42c5614c8418f34fd986e33d90be9e34d0: Status 404 returned error can't find the container with id 842ea7f7aa13c414c50b554ba5482c42c5614c8418f34fd986e33d90be9e34d0
Oct 30 12:56:25.330958 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:25.330930    2573 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9a20c40d_f5f4_4d96_85b6_be69a6cd1828.slice/crio-4ca8262c580200ac1c52c3d3783741b6914f7de30bbb7656c13e92eb6b55620e WatchSource:0}: Error finding container 4ca8262c580200ac1c52c3d3783741b6914f7de30bbb7656c13e92eb6b55620e: Status 404 returned error can't find the container with id 4ca8262c580200ac1c52c3d3783741b6914f7de30bbb7656c13e92eb6b55620e
Oct 30 12:56:25.333526 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.333506    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-m4p7x"
Oct 30 12:56:25.334642 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.334620    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-kb9lc"
Oct 30 12:56:25.341394 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.341375    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-h8q28"
Oct 30 12:56:25.349432 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.349410    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-8mfkz"
Oct 30 12:56:25.357728 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.357541    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-zth9k"
Oct 30 12:56:25.367207 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:25.367136    2573 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9962cf52_bd50_4a97_a699_4da03085eb81.slice/crio-7950ed323f5ca09de76caa701be569bfe27144ce3c8f0be537a097a4d2ec2e47 WatchSource:0}: Error finding container 7950ed323f5ca09de76caa701be569bfe27144ce3c8f0be537a097a4d2ec2e47: Status 404 returned error can't find the container with id 7950ed323f5ca09de76caa701be569bfe27144ce3c8f0be537a097a4d2ec2e47
Oct 30 12:56:25.369200 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:25.369177    2573 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podac1dc607_3072_4dbc_87e9_db316bf42dff.slice/crio-d32290f4009892eecbff7fd69950ecfd46024944d1fb20ad7609255ba446c867 WatchSource:0}: Error finding container d32290f4009892eecbff7fd69950ecfd46024944d1fb20ad7609255ba446c867: Status 404 returned error can't find the container with id d32290f4009892eecbff7fd69950ecfd46024944d1fb20ad7609255ba446c867
Oct 30 12:56:25.370114 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:25.369995    2573 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod16402666_f516_49d1_be0d_f6ab7234e637.slice/crio-72c23193bad92438cb473b27beb7f3f98a9a10064ec52310d78c6637bae06e28 WatchSource:0}: Error finding container 72c23193bad92438cb473b27beb7f3f98a9a10064ec52310d78c6637bae06e28: Status 404 returned error can't find the container with id 72c23193bad92438cb473b27beb7f3f98a9a10064ec52310d78c6637bae06e28
Oct 30 12:56:25.374133 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:25.374067    2573 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod51d01732_87a4_42b0_a6d5_a3fac689d7f6.slice/crio-8a35b228406b2d0201049df742444e4ce6aceda6624f683a21b2f712acc51808 WatchSource:0}: Error finding container 8a35b228406b2d0201049df742444e4ce6aceda6624f683a21b2f712acc51808: Status 404 returned error can't find the container with id 8a35b228406b2d0201049df742444e4ce6aceda6624f683a21b2f712acc51808
Oct 30 12:56:25.374398 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.374355    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-7rwcq"
Oct 30 12:56:25.375777 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.375755    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-4lfbk"
Oct 30 12:56:25.386458 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:25.386433    2573 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4817ec63_f4d5_4420_9417_148346ff8bb5.slice/crio-542a1406489aeed500a43e313ecb063b343fb53aa6b064ac09cbc4346db8987c WatchSource:0}: Error finding container 542a1406489aeed500a43e313ecb063b343fb53aa6b064ac09cbc4346db8987c: Status 404 returned error can't find the container with id 542a1406489aeed500a43e313ecb063b343fb53aa6b064ac09cbc4346db8987c
Oct 30 12:56:25.388840 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:25.388816    2573 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeca286e8_0b02_42af_9d15_1d8670b7362c.slice/crio-921c5569e0b7dd8f4c9893322f95435bbff7ce00371964f243fd87193d22af85 WatchSource:0}: Error finding container 921c5569e0b7dd8f4c9893322f95435bbff7ce00371964f243fd87193d22af85: Status 404 returned error can't find the container with id 921c5569e0b7dd8f4c9893322f95435bbff7ce00371964f243fd87193d22af85
Oct 30 12:56:25.795135 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.794889    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f660a256-a12d-4a4a-823d-4b0ff5dcb3da-original-pull-secret\") pod \"global-pull-secret-syncer-rpdcd\" (UID: \"f660a256-a12d-4a4a-823d-4b0ff5dcb3da\") " pod="kube-system/global-pull-secret-syncer-rpdcd"
Oct 30 12:56:25.795314 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.795202    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b11d3d97-ad88-4aa3-aaee-e83e3cf12f6d-metrics-certs\") pod \"network-metrics-daemon-p7tv6\" (UID: \"b11d3d97-ad88-4aa3-aaee-e83e3cf12f6d\") " pod="openshift-multus/network-metrics-daemon-p7tv6"
Oct 30 12:56:25.795314 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:25.795251    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9vr7r\" (UniqueName: \"kubernetes.io/projected/8d767e44-14df-4ea6-b391-bdf265773535-kube-api-access-9vr7r\") pod \"network-check-target-l5mdf\" (UID: \"8d767e44-14df-4ea6-b391-bdf265773535\") " pod="openshift-network-diagnostics/network-check-target-l5mdf"
Oct 30 12:56:25.795406 ip-10-0-142-84 kubenswrapper[2573]: E1030 12:56:25.795003    2573 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 30 12:56:25.795471 ip-10-0-142-84 kubenswrapper[2573]: E1030 12:56:25.795458    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f660a256-a12d-4a4a-823d-4b0ff5dcb3da-original-pull-secret podName:f660a256-a12d-4a4a-823d-4b0ff5dcb3da nodeName:}" failed. No retries permitted until 2026-10-30 12:56:26.795440011 +0000 UTC m=+4.195493371 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f660a256-a12d-4a4a-823d-4b0ff5dcb3da-original-pull-secret") pod "global-pull-secret-syncer-rpdcd" (UID: "f660a256-a12d-4a4a-823d-4b0ff5dcb3da") : object "kube-system"/"original-pull-secret" not registered
Oct 30 12:56:25.795799 ip-10-0-142-84 kubenswrapper[2573]: E1030 12:56:25.795778    2573 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 12:56:25.795849 ip-10-0-142-84 kubenswrapper[2573]: E1030 12:56:25.795827    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b11d3d97-ad88-4aa3-aaee-e83e3cf12f6d-metrics-certs podName:b11d3d97-ad88-4aa3-aaee-e83e3cf12f6d nodeName:}" failed. No retries permitted until 2026-10-30 12:56:26.795815627 +0000 UTC m=+4.195868967 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b11d3d97-ad88-4aa3-aaee-e83e3cf12f6d-metrics-certs") pod "network-metrics-daemon-p7tv6" (UID: "b11d3d97-ad88-4aa3-aaee-e83e3cf12f6d") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 12:56:25.795849 ip-10-0-142-84 kubenswrapper[2573]: E1030 12:56:25.795392    2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 30 12:56:25.795849 ip-10-0-142-84 kubenswrapper[2573]: E1030 12:56:25.795848    2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 30 12:56:25.795979 ip-10-0-142-84 kubenswrapper[2573]: E1030 12:56:25.795858    2573 projected.go:196] Error preparing data for projected volume kube-api-access-9vr7r for pod openshift-network-diagnostics/network-check-target-l5mdf: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 12:56:25.795979 ip-10-0-142-84 kubenswrapper[2573]: E1030 12:56:25.795882    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8d767e44-14df-4ea6-b391-bdf265773535-kube-api-access-9vr7r podName:8d767e44-14df-4ea6-b391-bdf265773535 nodeName:}" failed. No retries permitted until 2026-10-30 12:56:26.795875083 +0000 UTC m=+4.195928437 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-9vr7r" (UniqueName: "kubernetes.io/projected/8d767e44-14df-4ea6-b391-bdf265773535-kube-api-access-9vr7r") pod "network-check-target-l5mdf" (UID: "8d767e44-14df-4ea6-b391-bdf265773535") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 12:56:26.048061 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:26.047994    2573 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-10-29 12:51:25 +0000 UTC" deadline="2028-06-20 11:20:22.873519568 +0000 UTC"
Oct 30 12:56:26.048061 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:26.048059    2573 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14374h23m56.825464583s"
Oct 30 12:56:26.101707 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:26.101633    2573 generic.go:356] "Generic (PLEG): container finished" podID="6dab6bee2c58785dc24bb59bb1408683" containerID="3e0d219cdbabee434c71ad7be165744aeaddbbf7941f509aeff6755af5119382" exitCode=0
Oct 30 12:56:26.101892 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:26.101729    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-84.ec2.internal" event={"ID":"6dab6bee2c58785dc24bb59bb1408683","Type":"ContainerDied","Data":"3e0d219cdbabee434c71ad7be165744aeaddbbf7941f509aeff6755af5119382"}
Oct 30 12:56:26.106857 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:26.106758    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-4lfbk" event={"ID":"eca286e8-0b02-42af-9d15-1d8670b7362c","Type":"ContainerStarted","Data":"921c5569e0b7dd8f4c9893322f95435bbff7ce00371964f243fd87193d22af85"}
Oct 30 12:56:26.132151 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:26.132002    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7rwcq" event={"ID":"4817ec63-f4d5-4420-9417-148346ff8bb5","Type":"ContainerStarted","Data":"542a1406489aeed500a43e313ecb063b343fb53aa6b064ac09cbc4346db8987c"}
Oct 30 12:56:26.156415 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:26.156027    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-h8q28" event={"ID":"16402666-f516-49d1-be0d-f6ab7234e637","Type":"ContainerStarted","Data":"72c23193bad92438cb473b27beb7f3f98a9a10064ec52310d78c6637bae06e28"}
Oct 30 12:56:26.160812 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:26.160720    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-m4p7x" event={"ID":"9962cf52-bd50-4a97-a699-4da03085eb81","Type":"ContainerStarted","Data":"7950ed323f5ca09de76caa701be569bfe27144ce3c8f0be537a097a4d2ec2e47"}
Oct 30 12:56:26.175037 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:26.174984    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8bnjr" event={"ID":"a3ceb095-c75c-43b3-9de5-7dda644eddb1","Type":"ContainerStarted","Data":"842ea7f7aa13c414c50b554ba5482c42c5614c8418f34fd986e33d90be9e34d0"}
Oct 30 12:56:26.202332 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:26.201905    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-84.ec2.internal" event={"ID":"0c9cf3ec0e3ab50517e6f80b07fb5fb2","Type":"ContainerStarted","Data":"1c55f0b1d73742e538f35231d27757d818ca3987270e1134eaafd83ed0bbe70c"}
Oct 30 12:56:26.212704 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:26.212622    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-zth9k" event={"ID":"51d01732-87a4-42b0-a6d5-a3fac689d7f6","Type":"ContainerStarted","Data":"8a35b228406b2d0201049df742444e4ce6aceda6624f683a21b2f712acc51808"}
Oct 30 12:56:26.228238 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:26.228204    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-8mfkz" event={"ID":"56ff7ccd-573e-4bb2-a2b7-199a0346ed9f","Type":"ContainerStarted","Data":"038c1a289517d827e7023fa7bed469aef84b62c676f4d72ed6abefb9e8881903"}
Oct 30 12:56:26.249591 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:26.249466    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-kb9lc" event={"ID":"ac1dc607-3072-4dbc-87e9-db316bf42dff","Type":"ContainerStarted","Data":"d32290f4009892eecbff7fd69950ecfd46024944d1fb20ad7609255ba446c867"}
Oct 30 12:56:26.272332 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:26.272301    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-2l8x5" event={"ID":"9a20c40d-f5f4-4d96-85b6-be69a6cd1828","Type":"ContainerStarted","Data":"4ca8262c580200ac1c52c3d3783741b6914f7de30bbb7656c13e92eb6b55620e"}
Oct 30 12:56:26.804162 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:26.803123    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9vr7r\" (UniqueName: \"kubernetes.io/projected/8d767e44-14df-4ea6-b391-bdf265773535-kube-api-access-9vr7r\") pod \"network-check-target-l5mdf\" (UID: \"8d767e44-14df-4ea6-b391-bdf265773535\") " pod="openshift-network-diagnostics/network-check-target-l5mdf"
Oct 30 12:56:26.804162 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:26.803187    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f660a256-a12d-4a4a-823d-4b0ff5dcb3da-original-pull-secret\") pod \"global-pull-secret-syncer-rpdcd\" (UID: \"f660a256-a12d-4a4a-823d-4b0ff5dcb3da\") " pod="kube-system/global-pull-secret-syncer-rpdcd"
Oct 30 12:56:26.804162 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:26.803239    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b11d3d97-ad88-4aa3-aaee-e83e3cf12f6d-metrics-certs\") pod \"network-metrics-daemon-p7tv6\" (UID: \"b11d3d97-ad88-4aa3-aaee-e83e3cf12f6d\") " pod="openshift-multus/network-metrics-daemon-p7tv6"
Oct 30 12:56:26.804162 ip-10-0-142-84 kubenswrapper[2573]: E1030 12:56:26.803402    2573 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 12:56:26.804162 ip-10-0-142-84 kubenswrapper[2573]: E1030 12:56:26.803462    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b11d3d97-ad88-4aa3-aaee-e83e3cf12f6d-metrics-certs podName:b11d3d97-ad88-4aa3-aaee-e83e3cf12f6d nodeName:}" failed. No retries permitted until 2026-10-30 12:56:28.803443338 +0000 UTC m=+6.203496697 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b11d3d97-ad88-4aa3-aaee-e83e3cf12f6d-metrics-certs") pod "network-metrics-daemon-p7tv6" (UID: "b11d3d97-ad88-4aa3-aaee-e83e3cf12f6d") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 12:56:26.804162 ip-10-0-142-84 kubenswrapper[2573]: E1030 12:56:26.803588    2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 30 12:56:26.804162 ip-10-0-142-84 kubenswrapper[2573]: E1030 12:56:26.803608    2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 30 12:56:26.804162 ip-10-0-142-84 kubenswrapper[2573]: E1030 12:56:26.803620    2573 projected.go:196] Error preparing data for projected volume kube-api-access-9vr7r for pod openshift-network-diagnostics/network-check-target-l5mdf: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 12:56:26.804162 ip-10-0-142-84 kubenswrapper[2573]: E1030 12:56:26.803662    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8d767e44-14df-4ea6-b391-bdf265773535-kube-api-access-9vr7r podName:8d767e44-14df-4ea6-b391-bdf265773535 nodeName:}" failed. No retries permitted until 2026-10-30 12:56:28.803648914 +0000 UTC m=+6.203702268 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-9vr7r" (UniqueName: "kubernetes.io/projected/8d767e44-14df-4ea6-b391-bdf265773535-kube-api-access-9vr7r") pod "network-check-target-l5mdf" (UID: "8d767e44-14df-4ea6-b391-bdf265773535") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 12:56:26.804162 ip-10-0-142-84 kubenswrapper[2573]: E1030 12:56:26.803729    2573 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 30 12:56:26.804162 ip-10-0-142-84 kubenswrapper[2573]: E1030 12:56:26.803762    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f660a256-a12d-4a4a-823d-4b0ff5dcb3da-original-pull-secret podName:f660a256-a12d-4a4a-823d-4b0ff5dcb3da nodeName:}" failed. No retries permitted until 2026-10-30 12:56:28.803751077 +0000 UTC m=+6.203804423 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f660a256-a12d-4a4a-823d-4b0ff5dcb3da-original-pull-secret") pod "global-pull-secret-syncer-rpdcd" (UID: "f660a256-a12d-4a4a-823d-4b0ff5dcb3da") : object "kube-system"/"original-pull-secret" not registered
Oct 30 12:56:26.906210 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:26.903613    2573 reflector.go:446] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 12:56:27.048706 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:27.048607    2573 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-10-29 12:51:25 +0000 UTC" deadline="2028-06-27 05:36:26.370692142 +0000 UTC"
Oct 30 12:56:27.048706 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:27.048674    2573 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14536h39m59.322023178s"
Oct 30 12:56:27.057851 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:27.053439    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rpdcd"
Oct 30 12:56:27.057851 ip-10-0-142-84 kubenswrapper[2573]: E1030 12:56:27.053576    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rpdcd" podUID="f660a256-a12d-4a4a-823d-4b0ff5dcb3da"
Oct 30 12:56:27.057851 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:27.054020    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p7tv6"
Oct 30 12:56:27.057851 ip-10-0-142-84 kubenswrapper[2573]: E1030 12:56:27.054129    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p7tv6" podUID="b11d3d97-ad88-4aa3-aaee-e83e3cf12f6d"
Oct 30 12:56:27.057851 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:27.054212    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l5mdf"
Oct 30 12:56:27.057851 ip-10-0-142-84 kubenswrapper[2573]: E1030 12:56:27.054304    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l5mdf" podUID="8d767e44-14df-4ea6-b391-bdf265773535"
Oct 30 12:56:27.284781 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:27.284745    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-84.ec2.internal" event={"ID":"6dab6bee2c58785dc24bb59bb1408683","Type":"ContainerStarted","Data":"39edb1d73c7239f611e2348acaf1783d095ee48b8a070c9406531ebc5b9e703d"}
Oct 30 12:56:27.327435 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:27.326689    2573 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-84.ec2.internal" podStartSLOduration=3.3266694980000002 podStartE2EDuration="3.326669498s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 12:56:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 12:56:26.241694084 +0000 UTC m=+3.641747449" watchObservedRunningTime="2026-10-30 12:56:27.326669498 +0000 UTC m=+4.726722862"
Oct 30 12:56:27.327435 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:27.326937    2573 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-84.ec2.internal" podStartSLOduration=3.326926954 podStartE2EDuration="3.326926954s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 12:56:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 12:56:27.32592676 +0000 UTC m=+4.725980125" watchObservedRunningTime="2026-10-30 12:56:27.326926954 +0000 UTC m=+4.726980319"
Oct 30 12:56:28.821606 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:28.821568    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f660a256-a12d-4a4a-823d-4b0ff5dcb3da-original-pull-secret\") pod \"global-pull-secret-syncer-rpdcd\" (UID: \"f660a256-a12d-4a4a-823d-4b0ff5dcb3da\") " pod="kube-system/global-pull-secret-syncer-rpdcd"
Oct 30 12:56:28.822093 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:28.821643    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b11d3d97-ad88-4aa3-aaee-e83e3cf12f6d-metrics-certs\") pod \"network-metrics-daemon-p7tv6\" (UID: \"b11d3d97-ad88-4aa3-aaee-e83e3cf12f6d\") " pod="openshift-multus/network-metrics-daemon-p7tv6"
Oct 30 12:56:28.822093 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:28.821683    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9vr7r\" (UniqueName: \"kubernetes.io/projected/8d767e44-14df-4ea6-b391-bdf265773535-kube-api-access-9vr7r\") pod \"network-check-target-l5mdf\" (UID: \"8d767e44-14df-4ea6-b391-bdf265773535\") " pod="openshift-network-diagnostics/network-check-target-l5mdf"
Oct 30 12:56:28.822093 ip-10-0-142-84 kubenswrapper[2573]: E1030 12:56:28.821823    2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 30 12:56:28.822093 ip-10-0-142-84 kubenswrapper[2573]: E1030 12:56:28.821837    2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 30 12:56:28.822093 ip-10-0-142-84 kubenswrapper[2573]: E1030 12:56:28.821846    2573 projected.go:196] Error preparing data for projected volume kube-api-access-9vr7r for pod openshift-network-diagnostics/network-check-target-l5mdf: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 12:56:28.822093 ip-10-0-142-84 kubenswrapper[2573]: E1030 12:56:28.821898    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8d767e44-14df-4ea6-b391-bdf265773535-kube-api-access-9vr7r podName:8d767e44-14df-4ea6-b391-bdf265773535 nodeName:}" failed. No retries permitted until 2026-10-30 12:56:32.821879201 +0000 UTC m=+10.221932539 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-9vr7r" (UniqueName: "kubernetes.io/projected/8d767e44-14df-4ea6-b391-bdf265773535-kube-api-access-9vr7r") pod "network-check-target-l5mdf" (UID: "8d767e44-14df-4ea6-b391-bdf265773535") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 12:56:28.822469 ip-10-0-142-84 kubenswrapper[2573]: E1030 12:56:28.822248    2573 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 30 12:56:28.822469 ip-10-0-142-84 kubenswrapper[2573]: E1030 12:56:28.822294    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f660a256-a12d-4a4a-823d-4b0ff5dcb3da-original-pull-secret podName:f660a256-a12d-4a4a-823d-4b0ff5dcb3da nodeName:}" failed. No retries permitted until 2026-10-30 12:56:32.822277852 +0000 UTC m=+10.222331196 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f660a256-a12d-4a4a-823d-4b0ff5dcb3da-original-pull-secret") pod "global-pull-secret-syncer-rpdcd" (UID: "f660a256-a12d-4a4a-823d-4b0ff5dcb3da") : object "kube-system"/"original-pull-secret" not registered
Oct 30 12:56:28.822469 ip-10-0-142-84 kubenswrapper[2573]: E1030 12:56:28.822390    2573 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 12:56:28.822469 ip-10-0-142-84 kubenswrapper[2573]: E1030 12:56:28.822436    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b11d3d97-ad88-4aa3-aaee-e83e3cf12f6d-metrics-certs podName:b11d3d97-ad88-4aa3-aaee-e83e3cf12f6d nodeName:}" failed. No retries permitted until 2026-10-30 12:56:32.822422182 +0000 UTC m=+10.222475526 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b11d3d97-ad88-4aa3-aaee-e83e3cf12f6d-metrics-certs") pod "network-metrics-daemon-p7tv6" (UID: "b11d3d97-ad88-4aa3-aaee-e83e3cf12f6d") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 12:56:29.055952 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:29.055484    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rpdcd"
Oct 30 12:56:29.055952 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:29.055501    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p7tv6"
Oct 30 12:56:29.055952 ip-10-0-142-84 kubenswrapper[2573]: E1030 12:56:29.055603    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rpdcd" podUID="f660a256-a12d-4a4a-823d-4b0ff5dcb3da"
Oct 30 12:56:29.055952 ip-10-0-142-84 kubenswrapper[2573]: E1030 12:56:29.055754    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p7tv6" podUID="b11d3d97-ad88-4aa3-aaee-e83e3cf12f6d"
Oct 30 12:56:29.055952 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:29.055829    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l5mdf"
Oct 30 12:56:29.055952 ip-10-0-142-84 kubenswrapper[2573]: E1030 12:56:29.055918    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l5mdf" podUID="8d767e44-14df-4ea6-b391-bdf265773535"
Oct 30 12:56:29.302288 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:29.301449    2573 generic.go:356] "Generic (PLEG): container finished" podID="9962cf52-bd50-4a97-a699-4da03085eb81" containerID="72b296697f032c3f34defef4af30184863991ef5228a2a1a619ee56ef45f1241" exitCode=0
Oct 30 12:56:29.302288 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:29.301500    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-m4p7x" event={"ID":"9962cf52-bd50-4a97-a699-4da03085eb81","Type":"ContainerDied","Data":"72b296697f032c3f34defef4af30184863991ef5228a2a1a619ee56ef45f1241"}
Oct 30 12:56:30.309789 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:30.309741    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-zth9k" event={"ID":"51d01732-87a4-42b0-a6d5-a3fac689d7f6","Type":"ContainerStarted","Data":"c626386311f053e75cf9b00cd916baf76399c23e8ad02028477a4d1aa01091c6"}
Oct 30 12:56:31.056727 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:31.055895    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rpdcd"
Oct 30 12:56:31.056727 ip-10-0-142-84 kubenswrapper[2573]: E1030 12:56:31.056035    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rpdcd" podUID="f660a256-a12d-4a4a-823d-4b0ff5dcb3da"
Oct 30 12:56:31.056727 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:31.056418    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p7tv6"
Oct 30 12:56:31.056727 ip-10-0-142-84 kubenswrapper[2573]: E1030 12:56:31.056539    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p7tv6" podUID="b11d3d97-ad88-4aa3-aaee-e83e3cf12f6d"
Oct 30 12:56:31.056727 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:31.056598    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l5mdf"
Oct 30 12:56:31.056727 ip-10-0-142-84 kubenswrapper[2573]: E1030 12:56:31.056666    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l5mdf" podUID="8d767e44-14df-4ea6-b391-bdf265773535"
Oct 30 12:56:31.326087 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:31.326033    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-h8q28" event={"ID":"16402666-f516-49d1-be0d-f6ab7234e637","Type":"ContainerStarted","Data":"b53e392b7f9ed852e59ca36bdd8f952ca7c8c77efa8dd203a25bb8d4ada87b4d"}
Oct 30 12:56:31.363448 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:31.363394    2573 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/konnectivity-agent-zth9k" podStartSLOduration=3.420725011 podStartE2EDuration="7.363373917s" totalImagesPullingTime="3.942648906s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 12:56:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 12:56:30.351079194 +0000 UTC m=+7.751132559" watchObservedRunningTime="2026-10-30 12:56:31.363373917 +0000 UTC m=+8.763427279"
Oct 30 12:56:31.363668 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:31.363630    2573 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-image-registry/node-ca-h8q28" podStartSLOduration=3.168861774 podStartE2EDuration="8.363621728s" totalImagesPullingTime="5.194759954s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 12:56:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 12:56:31.362983433 +0000 UTC m=+8.763036794" watchObservedRunningTime="2026-10-30 12:56:31.363621728 +0000 UTC m=+8.763675088"
Oct 30 12:56:32.330495 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:32.330457    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8bnjr" event={"ID":"a3ceb095-c75c-43b3-9de5-7dda644eddb1","Type":"ContainerStarted","Data":"79cece91eacc9bc781d221975bad2c8ab69c33d12ca399138411f4efb82abebb"}
Oct 30 12:56:32.857502 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:32.857469    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b11d3d97-ad88-4aa3-aaee-e83e3cf12f6d-metrics-certs\") pod \"network-metrics-daemon-p7tv6\" (UID: \"b11d3d97-ad88-4aa3-aaee-e83e3cf12f6d\") " pod="openshift-multus/network-metrics-daemon-p7tv6"
Oct 30 12:56:32.857621 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:32.857528    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9vr7r\" (UniqueName: \"kubernetes.io/projected/8d767e44-14df-4ea6-b391-bdf265773535-kube-api-access-9vr7r\") pod \"network-check-target-l5mdf\" (UID: \"8d767e44-14df-4ea6-b391-bdf265773535\") " pod="openshift-network-diagnostics/network-check-target-l5mdf"
Oct 30 12:56:32.857621 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:32.857578    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f660a256-a12d-4a4a-823d-4b0ff5dcb3da-original-pull-secret\") pod \"global-pull-secret-syncer-rpdcd\" (UID: \"f660a256-a12d-4a4a-823d-4b0ff5dcb3da\") " pod="kube-system/global-pull-secret-syncer-rpdcd"
Oct 30 12:56:32.857724 ip-10-0-142-84 kubenswrapper[2573]: E1030 12:56:32.857639    2573 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 12:56:32.857724 ip-10-0-142-84 kubenswrapper[2573]: E1030 12:56:32.857703    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b11d3d97-ad88-4aa3-aaee-e83e3cf12f6d-metrics-certs podName:b11d3d97-ad88-4aa3-aaee-e83e3cf12f6d nodeName:}" failed. No retries permitted until 2026-10-30 12:56:40.857684082 +0000 UTC m=+18.257737435 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b11d3d97-ad88-4aa3-aaee-e83e3cf12f6d-metrics-certs") pod "network-metrics-daemon-p7tv6" (UID: "b11d3d97-ad88-4aa3-aaee-e83e3cf12f6d") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 12:56:32.857724 ip-10-0-142-84 kubenswrapper[2573]: E1030 12:56:32.857703    2573 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 30 12:56:32.857887 ip-10-0-142-84 kubenswrapper[2573]: E1030 12:56:32.857742    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f660a256-a12d-4a4a-823d-4b0ff5dcb3da-original-pull-secret podName:f660a256-a12d-4a4a-823d-4b0ff5dcb3da nodeName:}" failed. No retries permitted until 2026-10-30 12:56:40.857731622 +0000 UTC m=+18.257784975 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f660a256-a12d-4a4a-823d-4b0ff5dcb3da-original-pull-secret") pod "global-pull-secret-syncer-rpdcd" (UID: "f660a256-a12d-4a4a-823d-4b0ff5dcb3da") : object "kube-system"/"original-pull-secret" not registered
Oct 30 12:56:32.857887 ip-10-0-142-84 kubenswrapper[2573]: E1030 12:56:32.857776    2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 30 12:56:32.857887 ip-10-0-142-84 kubenswrapper[2573]: E1030 12:56:32.857792    2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 30 12:56:32.857887 ip-10-0-142-84 kubenswrapper[2573]: E1030 12:56:32.857804    2573 projected.go:196] Error preparing data for projected volume kube-api-access-9vr7r for pod openshift-network-diagnostics/network-check-target-l5mdf: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 12:56:32.857887 ip-10-0-142-84 kubenswrapper[2573]: E1030 12:56:32.857852    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8d767e44-14df-4ea6-b391-bdf265773535-kube-api-access-9vr7r podName:8d767e44-14df-4ea6-b391-bdf265773535 nodeName:}" failed. No retries permitted until 2026-10-30 12:56:40.857835234 +0000 UTC m=+18.257888574 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-9vr7r" (UniqueName: "kubernetes.io/projected/8d767e44-14df-4ea6-b391-bdf265773535-kube-api-access-9vr7r") pod "network-check-target-l5mdf" (UID: "8d767e44-14df-4ea6-b391-bdf265773535") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 12:56:32.995815 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:32.995782    2573 plugin_watcher.go:201] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Oct 30 12:56:33.029573 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:33.029486    2573 reconciler.go:167] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-10-30T12:56:32.995801507Z","UUID":"e238eb5f-1bef-4f4c-bfea-71b587c91ddb","Handler":null,"Name":"","Endpoint":""}
Oct 30 12:56:33.034946 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:33.031399    2573 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Oct 30 12:56:33.034946 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:33.031426    2573 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Oct 30 12:56:33.061934 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:33.061134    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l5mdf"
Oct 30 12:56:33.061934 ip-10-0-142-84 kubenswrapper[2573]: E1030 12:56:33.061283    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l5mdf" podUID="8d767e44-14df-4ea6-b391-bdf265773535"
Oct 30 12:56:33.061934 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:33.061690    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rpdcd"
Oct 30 12:56:33.061934 ip-10-0-142-84 kubenswrapper[2573]: E1030 12:56:33.061810    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rpdcd" podUID="f660a256-a12d-4a4a-823d-4b0ff5dcb3da"
Oct 30 12:56:33.062237 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:33.061977    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p7tv6"
Oct 30 12:56:33.062642 ip-10-0-142-84 kubenswrapper[2573]: E1030 12:56:33.062545    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p7tv6" podUID="b11d3d97-ad88-4aa3-aaee-e83e3cf12f6d"
Oct 30 12:56:33.334115 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:33.334082    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-2l8x5" event={"ID":"9a20c40d-f5f4-4d96-85b6-be69a6cd1828","Type":"ContainerStarted","Data":"f3152f5ff43529034ced9810b4795ab8632537f76fffa8ca841d98df2e60e4e0"}
Oct 30 12:56:33.339901 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:33.339847    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8bnjr" event={"ID":"a3ceb095-c75c-43b3-9de5-7dda644eddb1","Type":"ContainerStarted","Data":"bfc0791fa9e7898d9e4703bc3e0aef92e01e4b497cd038cc3127b85913f69784"}
Oct 30 12:56:33.376407 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:33.376351    2573 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/multus-2l8x5" podStartSLOduration=2.814333884 podStartE2EDuration="10.376334693s" totalImagesPullingTime="7.562000809s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 12:56:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 12:56:33.375474543 +0000 UTC m=+10.775527904" watchObservedRunningTime="2026-10-30 12:56:33.376334693 +0000 UTC m=+10.776388052"
Oct 30 12:56:33.684832 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:33.684158    2573 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-zth9k"
Oct 30 12:56:33.685179 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:33.685159    2573 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-zth9k"
Oct 30 12:56:34.344107 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:34.343860    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-4lfbk" event={"ID":"eca286e8-0b02-42af-9d15-1d8670b7362c","Type":"ContainerStarted","Data":"f1c3606e82d4484d85196e7987dc5b517d8b408dbd734c078befa16da6918d9b"}
Oct 30 12:56:34.346443 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:34.346411    2573 generic.go:356] "Generic (PLEG): container finished" podID="9962cf52-bd50-4a97-a699-4da03085eb81" containerID="afb2509b01924c7b14517f5a1a969d41bb17933bce75a633ed981515cf663afc" exitCode=0
Oct 30 12:56:34.346540 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:34.346497    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-m4p7x" event={"ID":"9962cf52-bd50-4a97-a699-4da03085eb81","Type":"ContainerDied","Data":"afb2509b01924c7b14517f5a1a969d41bb17933bce75a633ed981515cf663afc"}
Oct 30 12:56:34.350745 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:34.350717    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8bnjr" event={"ID":"a3ceb095-c75c-43b3-9de5-7dda644eddb1","Type":"ContainerStarted","Data":"5b844899d4d6c37fe0e6a24f00f61cbe871df1f061edc224b38ebf374d0ebad0"}
Oct 30 12:56:34.424460 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:34.424411    2573 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8bnjr" podStartSLOduration=3.164651364 podStartE2EDuration="11.424398178s" totalImagesPullingTime="8.259746814s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 12:56:23 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 12:56:34.423828824 +0000 UTC m=+11.823882184" watchObservedRunningTime="2026-10-30 12:56:34.424398178 +0000 UTC m=+11.824451545"
Oct 30 12:56:34.424678 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:34.424649    2573 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-dns/node-resolver-4lfbk" podStartSLOduration=2.245935222 podStartE2EDuration="10.424641938s" totalImagesPullingTime="8.178706716s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 12:56:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 12:56:34.383013605 +0000 UTC m=+11.783066966" watchObservedRunningTime="2026-10-30 12:56:34.424641938 +0000 UTC m=+11.824695298"
Oct 30 12:56:35.053942 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:35.053486    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rpdcd"
Oct 30 12:56:35.053942 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:35.053512    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p7tv6"
Oct 30 12:56:35.053942 ip-10-0-142-84 kubenswrapper[2573]: E1030 12:56:35.053616    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rpdcd" podUID="f660a256-a12d-4a4a-823d-4b0ff5dcb3da"
Oct 30 12:56:35.053942 ip-10-0-142-84 kubenswrapper[2573]: E1030 12:56:35.053690    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p7tv6" podUID="b11d3d97-ad88-4aa3-aaee-e83e3cf12f6d"
Oct 30 12:56:35.053942 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:35.053701    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l5mdf"
Oct 30 12:56:35.053942 ip-10-0-142-84 kubenswrapper[2573]: E1030 12:56:35.053757    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l5mdf" podUID="8d767e44-14df-4ea6-b391-bdf265773535"
Oct 30 12:56:35.354984 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:35.354900    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-kb9lc" event={"ID":"ac1dc607-3072-4dbc-87e9-db316bf42dff","Type":"ContainerStarted","Data":"4c98f2409b08ea5f8b82034ee6149618effd836c083f302c40eb20aad9d37c7c"}
Oct 30 12:56:35.358542 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:35.358507    2573 generic.go:356] "Generic (PLEG): container finished" podID="9962cf52-bd50-4a97-a699-4da03085eb81" containerID="caf5c243de766fef8c22ec79680a8f66b98b27dd428288c4a1c2a7d6ddbaf20d" exitCode=0
Oct 30 12:56:35.359903 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:35.359520    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-m4p7x" event={"ID":"9962cf52-bd50-4a97-a699-4da03085eb81","Type":"ContainerDied","Data":"caf5c243de766fef8c22ec79680a8f66b98b27dd428288c4a1c2a7d6ddbaf20d"}
Oct 30 12:56:35.393525 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:35.393480    2573 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-kb9lc" podStartSLOduration=4.487388724 podStartE2EDuration="12.393463073s" totalImagesPullingTime="7.906074349s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 12:56:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 12:56:35.392907373 +0000 UTC m=+12.792960735" watchObservedRunningTime="2026-10-30 12:56:35.393463073 +0000 UTC m=+12.793516452"
Oct 30 12:56:36.363522 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:36.363433    2573 generic.go:356] "Generic (PLEG): container finished" podID="9962cf52-bd50-4a97-a699-4da03085eb81" containerID="7e6aed8f682a0a1e80249063c69ade97107d96da9122079edaa214cb269d3359" exitCode=0
Oct 30 12:56:36.364182 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:36.363516    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-m4p7x" event={"ID":"9962cf52-bd50-4a97-a699-4da03085eb81","Type":"ContainerDied","Data":"7e6aed8f682a0a1e80249063c69ade97107d96da9122079edaa214cb269d3359"}
Oct 30 12:56:36.365595 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:36.365567    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-8mfkz" event={"ID":"56ff7ccd-573e-4bb2-a2b7-199a0346ed9f","Type":"ContainerStarted","Data":"c10b902490985a1f3a18f35ca37448b7bc9ae1d848b307f06e485de4d051e1ff"}
Oct 30 12:56:36.475352 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:36.475305    2573 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-8mfkz" podStartSLOduration=2.191037007 podStartE2EDuration="12.475291167s" totalImagesPullingTime="10.28425416s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 12:56:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 12:56:36.474950452 +0000 UTC m=+13.875003812" watchObservedRunningTime="2026-10-30 12:56:36.475291167 +0000 UTC m=+13.875344526"
Oct 30 12:56:37.053127 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:37.053093    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p7tv6"
Oct 30 12:56:37.053338 ip-10-0-142-84 kubenswrapper[2573]: E1030 12:56:37.053211    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p7tv6" podUID="b11d3d97-ad88-4aa3-aaee-e83e3cf12f6d"
Oct 30 12:56:37.053570 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:37.053555    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l5mdf"
Oct 30 12:56:37.053669 ip-10-0-142-84 kubenswrapper[2573]: E1030 12:56:37.053650    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l5mdf" podUID="8d767e44-14df-4ea6-b391-bdf265773535"
Oct 30 12:56:37.053793 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:37.053757    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rpdcd"
Oct 30 12:56:37.053914 ip-10-0-142-84 kubenswrapper[2573]: E1030 12:56:37.053884    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rpdcd" podUID="f660a256-a12d-4a4a-823d-4b0ff5dcb3da"
Oct 30 12:56:38.372782 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:38.372484    2573 generic.go:356] "Generic (PLEG): container finished" podID="9962cf52-bd50-4a97-a699-4da03085eb81" containerID="537f45f4a3e8b6659aa72e92aef20c8f0056d277c3831c358ed263d8d2c7629e" exitCode=0
Oct 30 12:56:38.372782 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:38.372564    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-m4p7x" event={"ID":"9962cf52-bd50-4a97-a699-4da03085eb81","Type":"ContainerDied","Data":"537f45f4a3e8b6659aa72e92aef20c8f0056d277c3831c358ed263d8d2c7629e"}
Oct 30 12:56:39.053929 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:39.053897    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p7tv6"
Oct 30 12:56:39.053929 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:39.053933    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rpdcd"
Oct 30 12:56:39.054143 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:39.053896    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l5mdf"
Oct 30 12:56:39.054143 ip-10-0-142-84 kubenswrapper[2573]: E1030 12:56:39.054039    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p7tv6" podUID="b11d3d97-ad88-4aa3-aaee-e83e3cf12f6d"
Oct 30 12:56:39.054143 ip-10-0-142-84 kubenswrapper[2573]: E1030 12:56:39.054108    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rpdcd" podUID="f660a256-a12d-4a4a-823d-4b0ff5dcb3da"
Oct 30 12:56:39.054274 ip-10-0-142-84 kubenswrapper[2573]: E1030 12:56:39.054178    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l5mdf" podUID="8d767e44-14df-4ea6-b391-bdf265773535"
Oct 30 12:56:39.378093 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:39.378012    2573 generic.go:356] "Generic (PLEG): container finished" podID="9962cf52-bd50-4a97-a699-4da03085eb81" containerID="f2f38a511535e38fe136cff8ceee2c1c7807609526121506d357f18d9af124a6" exitCode=0
Oct 30 12:56:39.378093 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:39.378063    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-m4p7x" event={"ID":"9962cf52-bd50-4a97-a699-4da03085eb81","Type":"ContainerDied","Data":"f2f38a511535e38fe136cff8ceee2c1c7807609526121506d357f18d9af124a6"}
Oct 30 12:56:40.383102 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:40.383071    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7rwcq" event={"ID":"4817ec63-f4d5-4420-9417-148346ff8bb5","Type":"ContainerStarted","Data":"502cd61f9e5e10b4367416b438b8c188ad72a4257b25dfc2f2610c9ddc24572b"}
Oct 30 12:56:40.383454 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:40.383186    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7rwcq" event={"ID":"4817ec63-f4d5-4420-9417-148346ff8bb5","Type":"ContainerStarted","Data":"c653fd6fbc9f7080dad8405336e3bc39e8d5ef1ae5cb3b776ca3d0937e4b0f9e"}
Oct 30 12:56:40.383454 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:40.383241    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7rwcq" event={"ID":"4817ec63-f4d5-4420-9417-148346ff8bb5","Type":"ContainerStarted","Data":"2be759e245b4624a50bbdb69afd0684367914b3c3c6c256839ae65e16ec50e3d"}
Oct 30 12:56:40.383454 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:40.383303    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7rwcq" event={"ID":"4817ec63-f4d5-4420-9417-148346ff8bb5","Type":"ContainerStarted","Data":"08ee986e2766a316e81d2c0cff624014a8da4ad51cb21a87ab4faa3f46ea35ed"}
Oct 30 12:56:40.383454 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:40.383351    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7rwcq" event={"ID":"4817ec63-f4d5-4420-9417-148346ff8bb5","Type":"ContainerStarted","Data":"f2e8cc8eb6037505353543bba8c1aa055225c856b0179157b390c86a69b7cc1c"}
Oct 30 12:56:40.386744 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:40.386720    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-m4p7x" event={"ID":"9962cf52-bd50-4a97-a699-4da03085eb81","Type":"ContainerStarted","Data":"426be555be8063b4a2c9c89fb48b4f6c1d111c12ab415e08ade07f63c63934a7"}
Oct 30 12:56:40.453782 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:40.453734    2573 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-m4p7x" podStartSLOduration=7.473372727 podStartE2EDuration="17.453721233s" totalImagesPullingTime="9.980348506s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 12:56:23 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 12:56:40.453249133 +0000 UTC m=+17.853302503" watchObservedRunningTime="2026-10-30 12:56:40.453721233 +0000 UTC m=+17.853774592"
Oct 30 12:56:40.919956 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:40.919913    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b11d3d97-ad88-4aa3-aaee-e83e3cf12f6d-metrics-certs\") pod \"network-metrics-daemon-p7tv6\" (UID: \"b11d3d97-ad88-4aa3-aaee-e83e3cf12f6d\") " pod="openshift-multus/network-metrics-daemon-p7tv6"
Oct 30 12:56:40.919956 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:40.919962    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9vr7r\" (UniqueName: \"kubernetes.io/projected/8d767e44-14df-4ea6-b391-bdf265773535-kube-api-access-9vr7r\") pod \"network-check-target-l5mdf\" (UID: \"8d767e44-14df-4ea6-b391-bdf265773535\") " pod="openshift-network-diagnostics/network-check-target-l5mdf"
Oct 30 12:56:40.920185 ip-10-0-142-84 kubenswrapper[2573]: E1030 12:56:40.920058    2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 30 12:56:40.920185 ip-10-0-142-84 kubenswrapper[2573]: E1030 12:56:40.920059    2573 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 12:56:40.920185 ip-10-0-142-84 kubenswrapper[2573]: E1030 12:56:40.920078    2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 30 12:56:40.920185 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:40.920077    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f660a256-a12d-4a4a-823d-4b0ff5dcb3da-original-pull-secret\") pod \"global-pull-secret-syncer-rpdcd\" (UID: \"f660a256-a12d-4a4a-823d-4b0ff5dcb3da\") " pod="kube-system/global-pull-secret-syncer-rpdcd"
Oct 30 12:56:40.920185 ip-10-0-142-84 kubenswrapper[2573]: E1030 12:56:40.920088    2573 projected.go:196] Error preparing data for projected volume kube-api-access-9vr7r for pod openshift-network-diagnostics/network-check-target-l5mdf: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 12:56:40.920185 ip-10-0-142-84 kubenswrapper[2573]: E1030 12:56:40.920123    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b11d3d97-ad88-4aa3-aaee-e83e3cf12f6d-metrics-certs podName:b11d3d97-ad88-4aa3-aaee-e83e3cf12f6d nodeName:}" failed. No retries permitted until 2026-10-30 12:56:56.920106703 +0000 UTC m=+34.320160058 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b11d3d97-ad88-4aa3-aaee-e83e3cf12f6d-metrics-certs") pod "network-metrics-daemon-p7tv6" (UID: "b11d3d97-ad88-4aa3-aaee-e83e3cf12f6d") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 12:56:40.920185 ip-10-0-142-84 kubenswrapper[2573]: E1030 12:56:40.920129    2573 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 30 12:56:40.920185 ip-10-0-142-84 kubenswrapper[2573]: E1030 12:56:40.920157    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8d767e44-14df-4ea6-b391-bdf265773535-kube-api-access-9vr7r podName:8d767e44-14df-4ea6-b391-bdf265773535 nodeName:}" failed. No retries permitted until 2026-10-30 12:56:56.920144076 +0000 UTC m=+34.320197413 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-9vr7r" (UniqueName: "kubernetes.io/projected/8d767e44-14df-4ea6-b391-bdf265773535-kube-api-access-9vr7r") pod "network-check-target-l5mdf" (UID: "8d767e44-14df-4ea6-b391-bdf265773535") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 12:56:40.920185 ip-10-0-142-84 kubenswrapper[2573]: E1030 12:56:40.920172    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f660a256-a12d-4a4a-823d-4b0ff5dcb3da-original-pull-secret podName:f660a256-a12d-4a4a-823d-4b0ff5dcb3da nodeName:}" failed. No retries permitted until 2026-10-30 12:56:56.920165236 +0000 UTC m=+34.320218573 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f660a256-a12d-4a4a-823d-4b0ff5dcb3da-original-pull-secret") pod "global-pull-secret-syncer-rpdcd" (UID: "f660a256-a12d-4a4a-823d-4b0ff5dcb3da") : object "kube-system"/"original-pull-secret" not registered
Oct 30 12:56:41.053625 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:41.053600    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p7tv6"
Oct 30 12:56:41.053735 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:41.053624    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rpdcd"
Oct 30 12:56:41.053735 ip-10-0-142-84 kubenswrapper[2573]: E1030 12:56:41.053693    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p7tv6" podUID="b11d3d97-ad88-4aa3-aaee-e83e3cf12f6d"
Oct 30 12:56:41.053828 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:41.053761    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l5mdf"
Oct 30 12:56:41.053865 ip-10-0-142-84 kubenswrapper[2573]: E1030 12:56:41.053844    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l5mdf" podUID="8d767e44-14df-4ea6-b391-bdf265773535"
Oct 30 12:56:41.053954 ip-10-0-142-84 kubenswrapper[2573]: E1030 12:56:41.053934    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rpdcd" podUID="f660a256-a12d-4a4a-823d-4b0ff5dcb3da"
Oct 30 12:56:41.391141 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:41.391107    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7rwcq" event={"ID":"4817ec63-f4d5-4420-9417-148346ff8bb5","Type":"ContainerStarted","Data":"70659aea5e520f3c655863455aef2fc262cdd5086420325dd62daa21a14abca8"}
Oct 30 12:56:43.053517 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:43.053490    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rpdcd"
Oct 30 12:56:43.053929 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:43.053623    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l5mdf"
Oct 30 12:56:43.053929 ip-10-0-142-84 kubenswrapper[2573]: E1030 12:56:43.053627    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rpdcd" podUID="f660a256-a12d-4a4a-823d-4b0ff5dcb3da"
Oct 30 12:56:43.053929 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:43.053653    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p7tv6"
Oct 30 12:56:43.053929 ip-10-0-142-84 kubenswrapper[2573]: E1030 12:56:43.053746    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l5mdf" podUID="8d767e44-14df-4ea6-b391-bdf265773535"
Oct 30 12:56:43.053929 ip-10-0-142-84 kubenswrapper[2573]: E1030 12:56:43.053844    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p7tv6" podUID="b11d3d97-ad88-4aa3-aaee-e83e3cf12f6d"
Oct 30 12:56:43.397812 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:43.397740    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7rwcq" event={"ID":"4817ec63-f4d5-4420-9417-148346ff8bb5","Type":"ContainerStarted","Data":"ac283cd1f32889cd1697af02edd210af6d1f0ffa5b9f066074fe0f06276abc57"}
Oct 30 12:56:45.053082 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:45.053055    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rpdcd"
Oct 30 12:56:45.053082 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:45.053067    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p7tv6"
Oct 30 12:56:45.053803 ip-10-0-142-84 kubenswrapper[2573]: E1030 12:56:45.053167    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rpdcd" podUID="f660a256-a12d-4a4a-823d-4b0ff5dcb3da"
Oct 30 12:56:45.053803 ip-10-0-142-84 kubenswrapper[2573]: E1030 12:56:45.053210    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p7tv6" podUID="b11d3d97-ad88-4aa3-aaee-e83e3cf12f6d"
Oct 30 12:56:45.053803 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:45.053248    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l5mdf"
Oct 30 12:56:45.053803 ip-10-0-142-84 kubenswrapper[2573]: E1030 12:56:45.053355    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l5mdf" podUID="8d767e44-14df-4ea6-b391-bdf265773535"
Oct 30 12:56:45.405152 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:45.405058    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7rwcq" event={"ID":"4817ec63-f4d5-4420-9417-148346ff8bb5","Type":"ContainerStarted","Data":"d824a4d37ef3501d9f66cedd8c9575c7d07922617f2cb9c125dd22b779bf79df"}
Oct 30 12:56:45.405345 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:45.405327    2573 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-7rwcq"
Oct 30 12:56:45.420626 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:45.420599    2573 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-7rwcq"
Oct 30 12:56:45.500494 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:45.500443    2573 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-7rwcq" podStartSLOduration=6.802263068 podStartE2EDuration="21.500430367s" totalImagesPullingTime="14.698167299s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 12:56:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 12:56:45.500225423 +0000 UTC m=+22.900278783" watchObservedRunningTime="2026-10-30 12:56:45.500430367 +0000 UTC m=+22.900483726"
Oct 30 12:56:46.407518 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:46.407371    2573 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-7rwcq"
Oct 30 12:56:46.407859 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:46.407569    2573 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-7rwcq"
Oct 30 12:56:46.422043 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:46.422018    2573 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-7rwcq"
Oct 30 12:56:46.627560 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:46.627530    2573 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-p7tv6"]
Oct 30 12:56:46.627713 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:46.627642    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p7tv6"
Oct 30 12:56:46.627768 ip-10-0-142-84 kubenswrapper[2573]: E1030 12:56:46.627733    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p7tv6" podUID="b11d3d97-ad88-4aa3-aaee-e83e3cf12f6d"
Oct 30 12:56:46.631558 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:46.631536    2573 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-rpdcd"]
Oct 30 12:56:46.631672 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:46.631624    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rpdcd"
Oct 30 12:56:46.631717 ip-10-0-142-84 kubenswrapper[2573]: E1030 12:56:46.631691    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rpdcd" podUID="f660a256-a12d-4a4a-823d-4b0ff5dcb3da"
Oct 30 12:56:46.638093 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:46.638070    2573 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-l5mdf"]
Oct 30 12:56:46.638199 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:46.638168    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l5mdf"
Oct 30 12:56:46.638301 ip-10-0-142-84 kubenswrapper[2573]: E1030 12:56:46.638280    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l5mdf" podUID="8d767e44-14df-4ea6-b391-bdf265773535"
Oct 30 12:56:48.053137 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:48.053111    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l5mdf"
Oct 30 12:56:48.053488 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:48.053180    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p7tv6"
Oct 30 12:56:48.053488 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:48.053281    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rpdcd"
Oct 30 12:56:48.053488 ip-10-0-142-84 kubenswrapper[2573]: E1030 12:56:48.053287    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l5mdf" podUID="8d767e44-14df-4ea6-b391-bdf265773535"
Oct 30 12:56:48.053488 ip-10-0-142-84 kubenswrapper[2573]: E1030 12:56:48.053373    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p7tv6" podUID="b11d3d97-ad88-4aa3-aaee-e83e3cf12f6d"
Oct 30 12:56:48.053488 ip-10-0-142-84 kubenswrapper[2573]: E1030 12:56:48.053439    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rpdcd" podUID="f660a256-a12d-4a4a-823d-4b0ff5dcb3da"
Oct 30 12:56:50.053826 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:50.053795    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p7tv6"
Oct 30 12:56:50.054419 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:50.053834    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rpdcd"
Oct 30 12:56:50.054419 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:50.053835    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l5mdf"
Oct 30 12:56:50.054419 ip-10-0-142-84 kubenswrapper[2573]: E1030 12:56:50.053915    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p7tv6" podUID="b11d3d97-ad88-4aa3-aaee-e83e3cf12f6d"
Oct 30 12:56:50.054419 ip-10-0-142-84 kubenswrapper[2573]: E1030 12:56:50.053977    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rpdcd" podUID="f660a256-a12d-4a4a-823d-4b0ff5dcb3da"
Oct 30 12:56:50.054419 ip-10-0-142-84 kubenswrapper[2573]: E1030 12:56:50.054074    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l5mdf" podUID="8d767e44-14df-4ea6-b391-bdf265773535"
Oct 30 12:56:52.053756 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:52.053725    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l5mdf"
Oct 30 12:56:52.054245 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:52.053825    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rpdcd"
Oct 30 12:56:52.054245 ip-10-0-142-84 kubenswrapper[2573]: E1030 12:56:52.053836    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l5mdf" podUID="8d767e44-14df-4ea6-b391-bdf265773535"
Oct 30 12:56:52.054245 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:52.053909    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p7tv6"
Oct 30 12:56:52.054245 ip-10-0-142-84 kubenswrapper[2573]: E1030 12:56:52.053984    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p7tv6" podUID="b11d3d97-ad88-4aa3-aaee-e83e3cf12f6d"
Oct 30 12:56:52.054245 ip-10-0-142-84 kubenswrapper[2573]: E1030 12:56:52.053907    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rpdcd" podUID="f660a256-a12d-4a4a-823d-4b0ff5dcb3da"
Oct 30 12:56:52.834201 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:52.834172    2573 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-84.ec2.internal" event="NodeReady"
Oct 30 12:56:52.834373 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:52.834309    2573 kubelet_node_status.go:477] "Fast updating node status as it just became ready"
Oct 30 12:56:52.882277 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:52.882241    2573 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-k4v7j"]
Oct 30 12:56:52.917014 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:52.916988    2573 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-h7brb"]
Oct 30 12:56:52.917178 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:52.917097    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kas-connection-checker-685895c7c9-k4v7j"
Oct 30 12:56:52.934534 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:52.934510    2573 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-xtfpg"]
Oct 30 12:56:52.934696 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:52.934677    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kas-connection-checker-685895c7c9-h7brb"
Oct 30 12:56:52.937670 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:52.937647    2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"openshift-service-ca.crt\""
Oct 30 12:56:52.937810 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:52.937675    2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"kube-root-ca.crt\""
Oct 30 12:56:52.948771 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:52.948745    2573 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-7c4986d449-7cfvt"]
Oct 30 12:56:52.948771 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:52.948744    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-xtfpg"
Oct 30 12:56:52.961366 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:52.961346    2573 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-4zl6m"]
Oct 30 12:56:52.961469 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:52.961459    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-7c4986d449-7cfvt"
Oct 30 12:56:52.975795 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:52.975776    2573 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\""
Oct 30 12:56:52.982095 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:52.982066    2573 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-69c87564f5-n6wjk"]
Oct 30 12:56:52.982202 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:52.982182    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kas-connection-checker-685895c7c9-4zl6m"
Oct 30 12:56:52.995667 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:52.995650    2573 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-884576c57-2sfj9"]
Oct 30 12:56:52.995794 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:52.995779    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-69c87564f5-n6wjk"
Oct 30 12:56:53.006498 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.006482    2573 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-544d749c8c-h42v2"]
Oct 30 12:56:53.006615 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.006598    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-884576c57-2sfj9"
Oct 30 12:56:53.011640 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.011620    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w2lch\" (UniqueName: \"kubernetes.io/projected/f07abb3f-8900-4fa5-8d35-c7ab1d35582d-kube-api-access-w2lch\") pod \"volume-data-source-validator-58c9f5f46d-xtfpg\" (UID: \"f07abb3f-8900-4fa5-8d35-c7ab1d35582d\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-xtfpg"
Oct 30 12:56:53.011755 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.011662    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-49z5c\" (UniqueName: \"kubernetes.io/projected/301eebcb-3d48-4eef-a0de-a1173e23ad65-kube-api-access-49z5c\") pod \"kas-connection-checker-685895c7c9-h7brb\" (UID: \"301eebcb-3d48-4eef-a0de-a1173e23ad65\") " pod="kube-system/kas-connection-checker-685895c7c9-h7brb"
Oct 30 12:56:53.011755 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.011739    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-swxqr\" (UniqueName: \"kubernetes.io/projected/0905782c-26ad-4dc8-93cf-d6f211ea7af5-kube-api-access-swxqr\") pod \"kas-connection-checker-685895c7c9-k4v7j\" (UID: \"0905782c-26ad-4dc8-93cf-d6f211ea7af5\") " pod="kube-system/kas-connection-checker-685895c7c9-k4v7j"
Oct 30 12:56:53.034737 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.034720    2573 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-plmkt"]
Oct 30 12:56:53.034848 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.034833    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-h42v2"
Oct 30 12:56:53.037579 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.037560    2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\""
Oct 30 12:56:53.037672 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.037590    2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Oct 30 12:56:53.037672 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.037594    2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Oct 30 12:56:53.037672 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.037644    2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\""
Oct 30 12:56:53.037806 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.037691    2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Oct 30 12:56:53.037806 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.037697    2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Oct 30 12:56:53.037806 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.037725    2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Oct 30 12:56:53.048818 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.048801    2573 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-86c478bb88-vsqgb"]
Oct 30 12:56:53.048911 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.048896    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-plmkt"
Oct 30 12:56:53.049604 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.049532    2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\""
Oct 30 12:56:53.061183 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.061164    2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Oct 30 12:56:53.061480 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.061273    2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Oct 30 12:56:53.063595 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.063580    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-86c478bb88-vsqgb"
Oct 30 12:56:53.067055 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.067041    2573 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-7dtcd"]
Oct 30 12:56:53.077599 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.077581    2573 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\""
Oct 30 12:56:53.077599 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.077593    2573 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Oct 30 12:56:53.077761 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.077599    2573 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Oct 30 12:56:53.077761 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.077583    2573 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\""
Oct 30 12:56:53.077761 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.077607    2573 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\""
Oct 30 12:56:53.077761 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.077709    2573 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-hp484\""
Oct 30 12:56:53.077887 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.077867    2573 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
Oct 30 12:56:53.079290 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.078301    2573 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Oct 30 12:56:53.079290 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.078601    2573 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-dockercfg-7lnpl\""
Oct 30 12:56:53.091080 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.091039    2573 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-7cd56b8dbc-8fqfr"]
Oct 30 12:56:53.091149 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.091139    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-7dtcd"
Oct 30 12:56:53.111293 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.111256    2573 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-649bdc978f-lkpqz"]
Oct 30 12:56:53.111361 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.111324    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-7cd56b8dbc-8fqfr"
Oct 30 12:56:53.112172 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.112149    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w2lch\" (UniqueName: \"kubernetes.io/projected/f07abb3f-8900-4fa5-8d35-c7ab1d35582d-kube-api-access-w2lch\") pod \"volume-data-source-validator-58c9f5f46d-xtfpg\" (UID: \"f07abb3f-8900-4fa5-8d35-c7ab1d35582d\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-xtfpg"
Oct 30 12:56:53.112274 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.112206    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-49z5c\" (UniqueName: \"kubernetes.io/projected/301eebcb-3d48-4eef-a0de-a1173e23ad65-kube-api-access-49z5c\") pod \"kas-connection-checker-685895c7c9-h7brb\" (UID: \"301eebcb-3d48-4eef-a0de-a1173e23ad65\") " pod="kube-system/kas-connection-checker-685895c7c9-h7brb"
Oct 30 12:56:53.112344 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.112277    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bd9e55af-64ad-4ffc-92c3-53b7bf654322-service-ca-bundle\") pod \"insights-operator-7c4986d449-7cfvt\" (UID: \"bd9e55af-64ad-4ffc-92c3-53b7bf654322\") " pod="openshift-insights/insights-operator-7c4986d449-7cfvt"
Oct 30 12:56:53.112344 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.112316    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qwmfv\" (UniqueName: \"kubernetes.io/projected/19b38c46-bfd4-4ed0-a764-f7b672f18cc6-kube-api-access-qwmfv\") pod \"cluster-monitoring-operator-884576c57-2sfj9\" (UID: \"19b38c46-bfd4-4ed0-a764-f7b672f18cc6\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-2sfj9"
Oct 30 12:56:53.112441 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.112363    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b86e8e45-3c2d-46b6-9ade-d390b56e4a75-registry-tls\") pod \"image-registry-69c87564f5-n6wjk\" (UID: \"b86e8e45-3c2d-46b6-9ade-d390b56e4a75\") " pod="openshift-image-registry/image-registry-69c87564f5-n6wjk"
Oct 30 12:56:53.112441 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.112390    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/85f4c399-0089-4399-86e2-069834f89286-serving-cert\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-plmkt\" (UID: \"85f4c399-0089-4399-86e2-069834f89286\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-plmkt"
Oct 30 12:56:53.112542 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.112480    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/85f4c399-0089-4399-86e2-069834f89286-config\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-plmkt\" (UID: \"85f4c399-0089-4399-86e2-069834f89286\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-plmkt"
Oct 30 12:56:53.112602 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.112583    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-swxqr\" (UniqueName: \"kubernetes.io/projected/0905782c-26ad-4dc8-93cf-d6f211ea7af5-kube-api-access-swxqr\") pod \"kas-connection-checker-685895c7c9-k4v7j\" (UID: \"0905782c-26ad-4dc8-93cf-d6f211ea7af5\") " pod="kube-system/kas-connection-checker-685895c7c9-k4v7j"
Oct 30 12:56:53.112652 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.112620    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b86e8e45-3c2d-46b6-9ade-d390b56e4a75-bound-sa-token\") pod \"image-registry-69c87564f5-n6wjk\" (UID: \"b86e8e45-3c2d-46b6-9ade-d390b56e4a75\") " pod="openshift-image-registry/image-registry-69c87564f5-n6wjk"
Oct 30 12:56:53.112755 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.112735    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l8pqr\" (UniqueName: \"kubernetes.io/projected/b86e8e45-3c2d-46b6-9ade-d390b56e4a75-kube-api-access-l8pqr\") pod \"image-registry-69c87564f5-n6wjk\" (UID: \"b86e8e45-3c2d-46b6-9ade-d390b56e4a75\") " pod="openshift-image-registry/image-registry-69c87564f5-n6wjk"
Oct 30 12:56:53.112809 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.112770    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/27c55f84-4e9a-44de-94ea-02258d6bc747-serving-cert\") pod \"service-ca-operator-544d749c8c-h42v2\" (UID: \"27c55f84-4e9a-44de-94ea-02258d6bc747\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-h42v2"
Oct 30 12:56:53.112809 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.112799    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bd9e55af-64ad-4ffc-92c3-53b7bf654322-tmp\") pod \"insights-operator-7c4986d449-7cfvt\" (UID: \"bd9e55af-64ad-4ffc-92c3-53b7bf654322\") " pod="openshift-insights/insights-operator-7c4986d449-7cfvt"
Oct 30 12:56:53.112902 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.112894    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/bd9e55af-64ad-4ffc-92c3-53b7bf654322-snapshots\") pod \"insights-operator-7c4986d449-7cfvt\" (UID: \"bd9e55af-64ad-4ffc-92c3-53b7bf654322\") " pod="openshift-insights/insights-operator-7c4986d449-7cfvt"
Oct 30 12:56:53.112948 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.112912    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/bd9e55af-64ad-4ffc-92c3-53b7bf654322-serving-cert\") pod \"insights-operator-7c4986d449-7cfvt\" (UID: \"bd9e55af-64ad-4ffc-92c3-53b7bf654322\") " pod="openshift-insights/insights-operator-7c4986d449-7cfvt"
Oct 30 12:56:53.112948 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.112929    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/b86e8e45-3c2d-46b6-9ade-d390b56e4a75-image-registry-private-configuration\") pod \"image-registry-69c87564f5-n6wjk\" (UID: \"b86e8e45-3c2d-46b6-9ade-d390b56e4a75\") " pod="openshift-image-registry/image-registry-69c87564f5-n6wjk"
Oct 30 12:56:53.112948 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.112945    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/b86e8e45-3c2d-46b6-9ade-d390b56e4a75-ca-trust-extracted\") pod \"image-registry-69c87564f5-n6wjk\" (UID: \"b86e8e45-3c2d-46b6-9ade-d390b56e4a75\") " pod="openshift-image-registry/image-registry-69c87564f5-n6wjk"
Oct 30 12:56:53.113087 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.112964    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/19b38c46-bfd4-4ed0-a764-f7b672f18cc6-telemetry-config\") pod \"cluster-monitoring-operator-884576c57-2sfj9\" (UID: \"19b38c46-bfd4-4ed0-a764-f7b672f18cc6\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-2sfj9"
Oct 30 12:56:53.113087 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.113009    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/b86e8e45-3c2d-46b6-9ade-d390b56e4a75-installation-pull-secrets\") pod \"image-registry-69c87564f5-n6wjk\" (UID: \"b86e8e45-3c2d-46b6-9ade-d390b56e4a75\") " pod="openshift-image-registry/image-registry-69c87564f5-n6wjk"
Oct 30 12:56:53.113087 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.113075    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/19b38c46-bfd4-4ed0-a764-f7b672f18cc6-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-2sfj9\" (UID: \"19b38c46-bfd4-4ed0-a764-f7b672f18cc6\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-2sfj9"
Oct 30 12:56:53.113280 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.113136    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/b86e8e45-3c2d-46b6-9ade-d390b56e4a75-registry-certificates\") pod \"image-registry-69c87564f5-n6wjk\" (UID: \"b86e8e45-3c2d-46b6-9ade-d390b56e4a75\") " pod="openshift-image-registry/image-registry-69c87564f5-n6wjk"
Oct 30 12:56:53.113280 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.113166    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/27c55f84-4e9a-44de-94ea-02258d6bc747-config\") pod \"service-ca-operator-544d749c8c-h42v2\" (UID: \"27c55f84-4e9a-44de-94ea-02258d6bc747\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-h42v2"
Oct 30 12:56:53.113354 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.113319    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6sjmw\" (UniqueName: \"kubernetes.io/projected/a4d58318-42ba-4a9e-9c74-43fe90baa4a7-kube-api-access-6sjmw\") pod \"kas-connection-checker-685895c7c9-4zl6m\" (UID: \"a4d58318-42ba-4a9e-9c74-43fe90baa4a7\") " pod="kube-system/kas-connection-checker-685895c7c9-4zl6m"
Oct 30 12:56:53.113403 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.113383    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rls7j\" (UniqueName: \"kubernetes.io/projected/85f4c399-0089-4399-86e2-069834f89286-kube-api-access-rls7j\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-plmkt\" (UID: \"85f4c399-0089-4399-86e2-069834f89286\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-plmkt"
Oct 30 12:56:53.113440 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.113425    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bd9e55af-64ad-4ffc-92c3-53b7bf654322-trusted-ca-bundle\") pod \"insights-operator-7c4986d449-7cfvt\" (UID: \"bd9e55af-64ad-4ffc-92c3-53b7bf654322\") " pod="openshift-insights/insights-operator-7c4986d449-7cfvt"
Oct 30 12:56:53.113484 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.113469    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/b86e8e45-3c2d-46b6-9ade-d390b56e4a75-trusted-ca\") pod \"image-registry-69c87564f5-n6wjk\" (UID: \"b86e8e45-3c2d-46b6-9ade-d390b56e4a75\") " pod="openshift-image-registry/image-registry-69c87564f5-n6wjk"
Oct 30 12:56:53.113527 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.113497    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hfnqv\" (UniqueName: \"kubernetes.io/projected/27c55f84-4e9a-44de-94ea-02258d6bc747-kube-api-access-hfnqv\") pod \"service-ca-operator-544d749c8c-h42v2\" (UID: \"27c55f84-4e9a-44de-94ea-02258d6bc747\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-h42v2"
Oct 30 12:56:53.113527 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.113522    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qt8c9\" (UniqueName: \"kubernetes.io/projected/bd9e55af-64ad-4ffc-92c3-53b7bf654322-kube-api-access-qt8c9\") pod \"insights-operator-7c4986d449-7cfvt\" (UID: \"bd9e55af-64ad-4ffc-92c3-53b7bf654322\") " pod="openshift-insights/insights-operator-7c4986d449-7cfvt"
Oct 30 12:56:53.125627 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.125611    2573 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-77c6ff895d-vw6c6"]
Oct 30 12:56:53.125744 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.125729    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-649bdc978f-lkpqz"
Oct 30 12:56:53.139407 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.139388    2573 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-g2ckh"]
Oct 30 12:56:53.139485 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.139470    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-77c6ff895d-vw6c6"
Oct 30 12:56:53.139576 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.139561    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l5mdf"
Oct 30 12:56:53.145387 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.145370    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w2lch\" (UniqueName: \"kubernetes.io/projected/f07abb3f-8900-4fa5-8d35-c7ab1d35582d-kube-api-access-w2lch\") pod \"volume-data-source-validator-58c9f5f46d-xtfpg\" (UID: \"f07abb3f-8900-4fa5-8d35-c7ab1d35582d\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-xtfpg"
Oct 30 12:56:53.146624 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.146600    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-swxqr\" (UniqueName: \"kubernetes.io/projected/0905782c-26ad-4dc8-93cf-d6f211ea7af5-kube-api-access-swxqr\") pod \"kas-connection-checker-685895c7c9-k4v7j\" (UID: \"0905782c-26ad-4dc8-93cf-d6f211ea7af5\") " pod="kube-system/kas-connection-checker-685895c7c9-k4v7j"
Oct 30 12:56:53.146996 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.146978    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-49z5c\" (UniqueName: \"kubernetes.io/projected/301eebcb-3d48-4eef-a0de-a1173e23ad65-kube-api-access-49z5c\") pod \"kas-connection-checker-685895c7c9-h7brb\" (UID: \"301eebcb-3d48-4eef-a0de-a1173e23ad65\") " pod="kube-system/kas-connection-checker-685895c7c9-h7brb"
Oct 30 12:56:53.147827 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.147803    2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Oct 30 12:56:53.148318 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.148252    2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\""
Oct 30 12:56:53.148693 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.148673    2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\""
Oct 30 12:56:53.148822 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.148675    2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Oct 30 12:56:53.148822 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.148698    2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\""
Oct 30 12:56:53.149004 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.148894    2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\""
Oct 30 12:56:53.149081 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.149061    2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\""
Oct 30 12:56:53.149178 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.149113    2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\""
Oct 30 12:56:53.149178 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.149154    2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-config\""
Oct 30 12:56:53.149178 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.149201    2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\""
Oct 30 12:56:53.149393 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.149375    2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\""
Oct 30 12:56:53.149466 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.149390    2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\""
Oct 30 12:56:53.149466 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.149456    2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
Oct 30 12:56:53.149568 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.149552    2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\""
Oct 30 12:56:53.150955 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.150941    2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\""
Oct 30 12:56:53.153760 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.153741    2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\""
Oct 30 12:56:53.154832 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.154814    2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Oct 30 12:56:53.154943 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.154929    2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Oct 30 12:56:53.158217 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.158202    2573 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-6f78d59f99-vdwgx"]
Oct 30 12:56:53.158336 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.158322    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-g2ckh"
Oct 30 12:56:53.162527 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.162508    2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\""
Oct 30 12:56:53.181873 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.181858    2573 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\""
Oct 30 12:56:53.182310 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.182292    2573 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\""
Oct 30 12:56:53.182394 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.182312    2573 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\""
Oct 30 12:56:53.182394 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.182330    2573 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\""
Oct 30 12:56:53.182394 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.182352    2573 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\""
Oct 30 12:56:53.182394 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.182389    2573 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-vg7bt\""
Oct 30 12:56:53.182557 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.182541    2573 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-bdf5s"]
Oct 30 12:56:53.182653 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.182640    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-6f78d59f99-vdwgx"
Oct 30 12:56:53.190882 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.190864    2573 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-h7brb"]
Oct 30 12:56:53.190958 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.190909    2573 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-k4v7j"]
Oct 30 12:56:53.190958 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.190928    2573 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-xtfpg"]
Oct 30 12:56:53.190958 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.190941    2573 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-86c478bb88-vsqgb"]
Oct 30 12:56:53.190958 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.190956    2573 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-7dtcd"]
Oct 30 12:56:53.191160 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.190974    2573 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-884576c57-2sfj9"]
Oct 30 12:56:53.191160 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.190974    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-bdf5s"
Oct 30 12:56:53.191160 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.190986    2573 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-69c87564f5-n6wjk"]
Oct 30 12:56:53.191160 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.191000    2573 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-plmkt"]
Oct 30 12:56:53.191160 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.191012    2573 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-7cd56b8dbc-8fqfr"]
Oct 30 12:56:53.191160 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.191023    2573 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-6f78d59f99-vdwgx"]
Oct 30 12:56:53.191160 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.191037    2573 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-g2ckh"]
Oct 30 12:56:53.191160 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.191052    2573 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-4zl6m"]
Oct 30 12:56:53.191160 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.191064    2573 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-544d749c8c-h42v2"]
Oct 30 12:56:53.191160 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.191076    2573 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-7c4986d449-7cfvt"]
Oct 30 12:56:53.191160 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.191093    2573 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-bdf5s"]
Oct 30 12:56:53.191160 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.191117    2573 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-649bdc978f-lkpqz"]
Oct 30 12:56:53.191160 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.191136    2573 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-77c6ff895d-vw6c6"]
Oct 30 12:56:53.214466 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.214448    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bd9e55af-64ad-4ffc-92c3-53b7bf654322-tmp\") pod \"insights-operator-7c4986d449-7cfvt\" (UID: \"bd9e55af-64ad-4ffc-92c3-53b7bf654322\") " pod="openshift-insights/insights-operator-7c4986d449-7cfvt"
Oct 30 12:56:53.214565 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.214474    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/b86e8e45-3c2d-46b6-9ade-d390b56e4a75-installation-pull-secrets\") pod \"image-registry-69c87564f5-n6wjk\" (UID: \"b86e8e45-3c2d-46b6-9ade-d390b56e4a75\") " pod="openshift-image-registry/image-registry-69c87564f5-n6wjk"
Oct 30 12:56:53.214565 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.214495    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/27c55f84-4e9a-44de-94ea-02258d6bc747-config\") pod \"service-ca-operator-544d749c8c-h42v2\" (UID: \"27c55f84-4e9a-44de-94ea-02258d6bc747\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-h42v2"
Oct 30 12:56:53.214565 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.214518    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/e1f629e6-6a96-444f-8329-e71bb98c9029-stats-auth\") pod \"router-default-649bdc978f-lkpqz\" (UID: \"e1f629e6-6a96-444f-8329-e71bb98c9029\") " pod="openshift-ingress/router-default-649bdc978f-lkpqz"
Oct 30 12:56:53.214565 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.214546    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/19b38c46-bfd4-4ed0-a764-f7b672f18cc6-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-2sfj9\" (UID: \"19b38c46-bfd4-4ed0-a764-f7b672f18cc6\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-2sfj9"
Oct 30 12:56:53.214725 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.214612    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/4883c26d-0877-4870-b0ee-406f5bfa8cf4-nginx-conf\") pod \"networking-console-plugin-86c478bb88-vsqgb\" (UID: \"4883c26d-0877-4870-b0ee-406f5bfa8cf4\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-vsqgb"
Oct 30 12:56:53.214725 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.214639    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bd9e55af-64ad-4ffc-92c3-53b7bf654322-trusted-ca-bundle\") pod \"insights-operator-7c4986d449-7cfvt\" (UID: \"bd9e55af-64ad-4ffc-92c3-53b7bf654322\") " pod="openshift-insights/insights-operator-7c4986d449-7cfvt"
Oct 30 12:56:53.214725 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.214663    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e1f629e6-6a96-444f-8329-e71bb98c9029-service-ca-bundle\") pod \"router-default-649bdc978f-lkpqz\" (UID: \"e1f629e6-6a96-444f-8329-e71bb98c9029\") " pod="openshift-ingress/router-default-649bdc978f-lkpqz"
Oct 30 12:56:53.214872 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.214769    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-config\" (UniqueName: \"kubernetes.io/configmap/23297e74-48f9-47b7-bb5a-80189bdbf2c4-samples-operator-config\") pod \"cluster-samples-operator-6ddb6c8b94-7dtcd\" (UID: \"23297e74-48f9-47b7-bb5a-80189bdbf2c4\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-7dtcd"
Oct 30 12:56:53.214872 ip-10-0-142-84 kubenswrapper[2573]: E1030 12:56:53.214795    2573 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Oct 30 12:56:53.214872 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.214802    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/d8930ee7-0ae0-48b5-8ce3-cb8820355cef-serving-cert\") pod \"console-operator-7cd56b8dbc-8fqfr\" (UID: \"d8930ee7-0ae0-48b5-8ce3-cb8820355cef\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-8fqfr"
Oct 30 12:56:53.214872 ip-10-0-142-84 kubenswrapper[2573]: E1030 12:56:53.214859    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/19b38c46-bfd4-4ed0-a764-f7b672f18cc6-cluster-monitoring-operator-tls podName:19b38c46-bfd4-4ed0-a764-f7b672f18cc6 nodeName:}" failed. No retries permitted until 2026-10-30 12:56:53.714841849 +0000 UTC m=+31.114895225 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/19b38c46-bfd4-4ed0-a764-f7b672f18cc6-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-2sfj9" (UID: "19b38c46-bfd4-4ed0-a764-f7b672f18cc6") : secret "cluster-monitoring-operator-tls" not found
Oct 30 12:56:53.215073 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.214901    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/85f4c399-0089-4399-86e2-069834f89286-serving-cert\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-plmkt\" (UID: \"85f4c399-0089-4399-86e2-069834f89286\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-plmkt"
Oct 30 12:56:53.215073 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.214943    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bd9e55af-64ad-4ffc-92c3-53b7bf654322-tmp\") pod \"insights-operator-7c4986d449-7cfvt\" (UID: \"bd9e55af-64ad-4ffc-92c3-53b7bf654322\") " pod="openshift-insights/insights-operator-7c4986d449-7cfvt"
Oct 30 12:56:53.215073 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.214952    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b86e8e45-3c2d-46b6-9ade-d390b56e4a75-registry-tls\") pod \"image-registry-69c87564f5-n6wjk\" (UID: \"b86e8e45-3c2d-46b6-9ade-d390b56e4a75\") " pod="openshift-image-registry/image-registry-69c87564f5-n6wjk"
Oct 30 12:56:53.215073 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.214978    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/4883c26d-0877-4870-b0ee-406f5bfa8cf4-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-vsqgb\" (UID: \"4883c26d-0877-4870-b0ee-406f5bfa8cf4\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-vsqgb"
Oct 30 12:56:53.215073 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.215008    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e1f629e6-6a96-444f-8329-e71bb98c9029-metrics-certs\") pod \"router-default-649bdc978f-lkpqz\" (UID: \"e1f629e6-6a96-444f-8329-e71bb98c9029\") " pod="openshift-ingress/router-default-649bdc978f-lkpqz"
Oct 30 12:56:53.215073 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.215031    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1c405568-cb88-4485-ae2c-3614cc31ced6-registry-tls\") pod \"image-registry-77c6ff895d-vw6c6\" (UID: \"1c405568-cb88-4485-ae2c-3614cc31ced6\") " pod="openshift-image-registry/image-registry-77c6ff895d-vw6c6"
Oct 30 12:56:53.215073 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.215054    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/1c405568-cb88-4485-ae2c-3614cc31ced6-trusted-ca\") pod \"image-registry-77c6ff895d-vw6c6\" (UID: \"1c405568-cb88-4485-ae2c-3614cc31ced6\") " pod="openshift-image-registry/image-registry-77c6ff895d-vw6c6"
Oct 30 12:56:53.215441 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.215076    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/e1f629e6-6a96-444f-8329-e71bb98c9029-default-certificate\") pod \"router-default-649bdc978f-lkpqz\" (UID: \"e1f629e6-6a96-444f-8329-e71bb98c9029\") " pod="openshift-ingress/router-default-649bdc978f-lkpqz"
Oct 30 12:56:53.215441 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.215099    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d8930ee7-0ae0-48b5-8ce3-cb8820355cef-trusted-ca\") pod \"console-operator-7cd56b8dbc-8fqfr\" (UID: \"d8930ee7-0ae0-48b5-8ce3-cb8820355cef\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-8fqfr"
Oct 30 12:56:53.215441 ip-10-0-142-84 kubenswrapper[2573]: E1030 12:56:53.215118    2573 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Oct 30 12:56:53.215441 ip-10-0-142-84 kubenswrapper[2573]: E1030 12:56:53.215131    2573 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-69c87564f5-n6wjk: secret "image-registry-tls" not found
Oct 30 12:56:53.215441 ip-10-0-142-84 kubenswrapper[2573]: E1030 12:56:53.215175    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b86e8e45-3c2d-46b6-9ade-d390b56e4a75-registry-tls podName:b86e8e45-3c2d-46b6-9ade-d390b56e4a75 nodeName:}" failed. No retries permitted until 2026-10-30 12:56:53.715161553 +0000 UTC m=+31.115214894 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/b86e8e45-3c2d-46b6-9ade-d390b56e4a75-registry-tls") pod "image-registry-69c87564f5-n6wjk" (UID: "b86e8e45-3c2d-46b6-9ade-d390b56e4a75") : secret "image-registry-tls" not found
Oct 30 12:56:53.215441 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.215214    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/27c55f84-4e9a-44de-94ea-02258d6bc747-config\") pod \"service-ca-operator-544d749c8c-h42v2\" (UID: \"27c55f84-4e9a-44de-94ea-02258d6bc747\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-h42v2"
Oct 30 12:56:53.215441 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.215219    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tltj6\" (UniqueName: \"kubernetes.io/projected/d8930ee7-0ae0-48b5-8ce3-cb8820355cef-kube-api-access-tltj6\") pod \"console-operator-7cd56b8dbc-8fqfr\" (UID: \"d8930ee7-0ae0-48b5-8ce3-cb8820355cef\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-8fqfr"
Oct 30 12:56:53.215441 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.215301    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/bd9e55af-64ad-4ffc-92c3-53b7bf654322-snapshots\") pod \"insights-operator-7c4986d449-7cfvt\" (UID: \"bd9e55af-64ad-4ffc-92c3-53b7bf654322\") " pod="openshift-insights/insights-operator-7c4986d449-7cfvt"
Oct 30 12:56:53.215441 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.215333    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/b86e8e45-3c2d-46b6-9ade-d390b56e4a75-image-registry-private-configuration\") pod \"image-registry-69c87564f5-n6wjk\" (UID: \"b86e8e45-3c2d-46b6-9ade-d390b56e4a75\") " pod="openshift-image-registry/image-registry-69c87564f5-n6wjk"
Oct 30 12:56:53.215441 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.215381    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/b86e8e45-3c2d-46b6-9ade-d390b56e4a75-ca-trust-extracted\") pod \"image-registry-69c87564f5-n6wjk\" (UID: \"b86e8e45-3c2d-46b6-9ade-d390b56e4a75\") " pod="openshift-image-registry/image-registry-69c87564f5-n6wjk"
Oct 30 12:56:53.215441 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.215412    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/bd9e55af-64ad-4ffc-92c3-53b7bf654322-serving-cert\") pod \"insights-operator-7c4986d449-7cfvt\" (UID: \"bd9e55af-64ad-4ffc-92c3-53b7bf654322\") " pod="openshift-insights/insights-operator-7c4986d449-7cfvt"
Oct 30 12:56:53.215441 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.215440    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/19b38c46-bfd4-4ed0-a764-f7b672f18cc6-telemetry-config\") pod \"cluster-monitoring-operator-884576c57-2sfj9\" (UID: \"19b38c46-bfd4-4ed0-a764-f7b672f18cc6\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-2sfj9"
Oct 30 12:56:53.216013 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.215473    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rls7j\" (UniqueName: \"kubernetes.io/projected/85f4c399-0089-4399-86e2-069834f89286-kube-api-access-rls7j\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-plmkt\" (UID: \"85f4c399-0089-4399-86e2-069834f89286\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-plmkt"
Oct 30 12:56:53.216013 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.215501    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/b86e8e45-3c2d-46b6-9ade-d390b56e4a75-registry-certificates\") pod \"image-registry-69c87564f5-n6wjk\" (UID: \"b86e8e45-3c2d-46b6-9ade-d390b56e4a75\") " pod="openshift-image-registry/image-registry-69c87564f5-n6wjk"
Oct 30 12:56:53.216013 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.215531    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/1c405568-cb88-4485-ae2c-3614cc31ced6-installation-pull-secrets\") pod \"image-registry-77c6ff895d-vw6c6\" (UID: \"1c405568-cb88-4485-ae2c-3614cc31ced6\") " pod="openshift-image-registry/image-registry-77c6ff895d-vw6c6"
Oct 30 12:56:53.216013 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.215560    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6sjmw\" (UniqueName: \"kubernetes.io/projected/a4d58318-42ba-4a9e-9c74-43fe90baa4a7-kube-api-access-6sjmw\") pod \"kas-connection-checker-685895c7c9-4zl6m\" (UID: \"a4d58318-42ba-4a9e-9c74-43fe90baa4a7\") " pod="kube-system/kas-connection-checker-685895c7c9-4zl6m"
Oct 30 12:56:53.216013 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.215584    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hfnqv\" (UniqueName: \"kubernetes.io/projected/27c55f84-4e9a-44de-94ea-02258d6bc747-kube-api-access-hfnqv\") pod \"service-ca-operator-544d749c8c-h42v2\" (UID: \"27c55f84-4e9a-44de-94ea-02258d6bc747\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-h42v2"
Oct 30 12:56:53.216013 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.215606    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bd9e55af-64ad-4ffc-92c3-53b7bf654322-trusted-ca-bundle\") pod \"insights-operator-7c4986d449-7cfvt\" (UID: \"bd9e55af-64ad-4ffc-92c3-53b7bf654322\") " pod="openshift-insights/insights-operator-7c4986d449-7cfvt"
Oct 30 12:56:53.216013 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.215616    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-phkm5\" (UniqueName: \"kubernetes.io/projected/e1f629e6-6a96-444f-8329-e71bb98c9029-kube-api-access-phkm5\") pod \"router-default-649bdc978f-lkpqz\" (UID: \"e1f629e6-6a96-444f-8329-e71bb98c9029\") " pod="openshift-ingress/router-default-649bdc978f-lkpqz"
Oct 30 12:56:53.216013 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.215639    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/b86e8e45-3c2d-46b6-9ade-d390b56e4a75-trusted-ca\") pod \"image-registry-69c87564f5-n6wjk\" (UID: \"b86e8e45-3c2d-46b6-9ade-d390b56e4a75\") " pod="openshift-image-registry/image-registry-69c87564f5-n6wjk"
Oct 30 12:56:53.216013 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.215732    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/1c405568-cb88-4485-ae2c-3614cc31ced6-image-registry-private-configuration\") pod \"image-registry-77c6ff895d-vw6c6\" (UID: \"1c405568-cb88-4485-ae2c-3614cc31ced6\") " pod="openshift-image-registry/image-registry-77c6ff895d-vw6c6"
Oct 30 12:56:53.216013 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.215789    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/1c405568-cb88-4485-ae2c-3614cc31ced6-ca-trust-extracted\") pod \"image-registry-77c6ff895d-vw6c6\" (UID: \"1c405568-cb88-4485-ae2c-3614cc31ced6\") " pod="openshift-image-registry/image-registry-77c6ff895d-vw6c6"
Oct 30 12:56:53.216013 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.215834    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/bd9e55af-64ad-4ffc-92c3-53b7bf654322-snapshots\") pod \"insights-operator-7c4986d449-7cfvt\" (UID: \"bd9e55af-64ad-4ffc-92c3-53b7bf654322\") " pod="openshift-insights/insights-operator-7c4986d449-7cfvt"
Oct 30 12:56:53.216575 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.216061    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/b86e8e45-3c2d-46b6-9ade-d390b56e4a75-ca-trust-extracted\") pod \"image-registry-69c87564f5-n6wjk\" (UID: \"b86e8e45-3c2d-46b6-9ade-d390b56e4a75\") " pod="openshift-image-registry/image-registry-69c87564f5-n6wjk"
Oct 30 12:56:53.216575 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.216213    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/b86e8e45-3c2d-46b6-9ade-d390b56e4a75-registry-certificates\") pod \"image-registry-69c87564f5-n6wjk\" (UID: \"b86e8e45-3c2d-46b6-9ade-d390b56e4a75\") " pod="openshift-image-registry/image-registry-69c87564f5-n6wjk"
Oct 30 12:56:53.216575 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.216293    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qt8c9\" (UniqueName: \"kubernetes.io/projected/bd9e55af-64ad-4ffc-92c3-53b7bf654322-kube-api-access-qt8c9\") pod \"insights-operator-7c4986d449-7cfvt\" (UID: \"bd9e55af-64ad-4ffc-92c3-53b7bf654322\") " pod="openshift-insights/insights-operator-7c4986d449-7cfvt"
Oct 30 12:56:53.216575 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.216389    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bd9e55af-64ad-4ffc-92c3-53b7bf654322-service-ca-bundle\") pod \"insights-operator-7c4986d449-7cfvt\" (UID: \"bd9e55af-64ad-4ffc-92c3-53b7bf654322\") " pod="openshift-insights/insights-operator-7c4986d449-7cfvt"
Oct 30 12:56:53.216575 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.216422    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jz42t\" (UniqueName: \"kubernetes.io/projected/23297e74-48f9-47b7-bb5a-80189bdbf2c4-kube-api-access-jz42t\") pod \"cluster-samples-operator-6ddb6c8b94-7dtcd\" (UID: \"23297e74-48f9-47b7-bb5a-80189bdbf2c4\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-7dtcd"
Oct 30 12:56:53.216575 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.216452    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qwmfv\" (UniqueName: \"kubernetes.io/projected/19b38c46-bfd4-4ed0-a764-f7b672f18cc6-kube-api-access-qwmfv\") pod \"cluster-monitoring-operator-884576c57-2sfj9\" (UID: \"19b38c46-bfd4-4ed0-a764-f7b672f18cc6\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-2sfj9"
Oct 30 12:56:53.216575 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.216481    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/1c405568-cb88-4485-ae2c-3614cc31ced6-registry-certificates\") pod \"image-registry-77c6ff895d-vw6c6\" (UID: \"1c405568-cb88-4485-ae2c-3614cc31ced6\") " pod="openshift-image-registry/image-registry-77c6ff895d-vw6c6"
Oct 30 12:56:53.216575 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.216507    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1c405568-cb88-4485-ae2c-3614cc31ced6-bound-sa-token\") pod \"image-registry-77c6ff895d-vw6c6\" (UID: \"1c405568-cb88-4485-ae2c-3614cc31ced6\") " pod="openshift-image-registry/image-registry-77c6ff895d-vw6c6"
Oct 30 12:56:53.216575 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.216536    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-54kft\" (UniqueName: \"kubernetes.io/projected/1c405568-cb88-4485-ae2c-3614cc31ced6-kube-api-access-54kft\") pod \"image-registry-77c6ff895d-vw6c6\" (UID: \"1c405568-cb88-4485-ae2c-3614cc31ced6\") " pod="openshift-image-registry/image-registry-77c6ff895d-vw6c6"
Oct 30 12:56:53.216575 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.216568    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/85f4c399-0089-4399-86e2-069834f89286-config\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-plmkt\" (UID: \"85f4c399-0089-4399-86e2-069834f89286\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-plmkt"
Oct 30 12:56:53.217051 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.216591    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/23297e74-48f9-47b7-bb5a-80189bdbf2c4-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-7dtcd\" (UID: \"23297e74-48f9-47b7-bb5a-80189bdbf2c4\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-7dtcd"
Oct 30 12:56:53.217051 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.216617    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/d8930ee7-0ae0-48b5-8ce3-cb8820355cef-config\") pod \"console-operator-7cd56b8dbc-8fqfr\" (UID: \"d8930ee7-0ae0-48b5-8ce3-cb8820355cef\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-8fqfr"
Oct 30 12:56:53.217051 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.216645    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b86e8e45-3c2d-46b6-9ade-d390b56e4a75-bound-sa-token\") pod \"image-registry-69c87564f5-n6wjk\" (UID: \"b86e8e45-3c2d-46b6-9ade-d390b56e4a75\") " pod="openshift-image-registry/image-registry-69c87564f5-n6wjk"
Oct 30 12:56:53.217051 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.216669    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l8pqr\" (UniqueName: \"kubernetes.io/projected/b86e8e45-3c2d-46b6-9ade-d390b56e4a75-kube-api-access-l8pqr\") pod \"image-registry-69c87564f5-n6wjk\" (UID: \"b86e8e45-3c2d-46b6-9ade-d390b56e4a75\") " pod="openshift-image-registry/image-registry-69c87564f5-n6wjk"
Oct 30 12:56:53.217051 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.216694    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/27c55f84-4e9a-44de-94ea-02258d6bc747-serving-cert\") pod \"service-ca-operator-544d749c8c-h42v2\" (UID: \"27c55f84-4e9a-44de-94ea-02258d6bc747\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-h42v2"
Oct 30 12:56:53.217051 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.216732    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/19b38c46-bfd4-4ed0-a764-f7b672f18cc6-telemetry-config\") pod \"cluster-monitoring-operator-884576c57-2sfj9\" (UID: \"19b38c46-bfd4-4ed0-a764-f7b672f18cc6\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-2sfj9"
Oct 30 12:56:53.217051 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.217017    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/b86e8e45-3c2d-46b6-9ade-d390b56e4a75-trusted-ca\") pod \"image-registry-69c87564f5-n6wjk\" (UID: \"b86e8e45-3c2d-46b6-9ade-d390b56e4a75\") " pod="openshift-image-registry/image-registry-69c87564f5-n6wjk"
Oct 30 12:56:53.217704 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.217677    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/b86e8e45-3c2d-46b6-9ade-d390b56e4a75-image-registry-private-configuration\") pod \"image-registry-69c87564f5-n6wjk\" (UID: \"b86e8e45-3c2d-46b6-9ade-d390b56e4a75\") " pod="openshift-image-registry/image-registry-69c87564f5-n6wjk"
Oct 30 12:56:53.217788 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.217730    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/b86e8e45-3c2d-46b6-9ade-d390b56e4a75-installation-pull-secrets\") pod \"image-registry-69c87564f5-n6wjk\" (UID: \"b86e8e45-3c2d-46b6-9ade-d390b56e4a75\") " pod="openshift-image-registry/image-registry-69c87564f5-n6wjk"
Oct 30 12:56:53.217788 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.217769    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/85f4c399-0089-4399-86e2-069834f89286-config\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-plmkt\" (UID: \"85f4c399-0089-4399-86e2-069834f89286\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-plmkt"
Oct 30 12:56:53.218005 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.217954    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/85f4c399-0089-4399-86e2-069834f89286-serving-cert\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-plmkt\" (UID: \"85f4c399-0089-4399-86e2-069834f89286\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-plmkt"
Oct 30 12:56:53.218100 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.218088    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bd9e55af-64ad-4ffc-92c3-53b7bf654322-service-ca-bundle\") pod \"insights-operator-7c4986d449-7cfvt\" (UID: \"bd9e55af-64ad-4ffc-92c3-53b7bf654322\") " pod="openshift-insights/insights-operator-7c4986d449-7cfvt"
Oct 30 12:56:53.219235 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.219214    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/27c55f84-4e9a-44de-94ea-02258d6bc747-serving-cert\") pod \"service-ca-operator-544d749c8c-h42v2\" (UID: \"27c55f84-4e9a-44de-94ea-02258d6bc747\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-h42v2"
Oct 30 12:56:53.219584 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.219566    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/bd9e55af-64ad-4ffc-92c3-53b7bf654322-serving-cert\") pod \"insights-operator-7c4986d449-7cfvt\" (UID: \"bd9e55af-64ad-4ffc-92c3-53b7bf654322\") " pod="openshift-insights/insights-operator-7c4986d449-7cfvt"
Oct 30 12:56:53.240555 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.240531    2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Oct 30 12:56:53.240650 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.240588    2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Oct 30 12:56:53.240704 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.240652    2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Oct 30 12:56:53.240920 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.240885    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kas-connection-checker-685895c7c9-k4v7j"
Oct 30 12:56:53.242543 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.242520    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6sjmw\" (UniqueName: \"kubernetes.io/projected/a4d58318-42ba-4a9e-9c74-43fe90baa4a7-kube-api-access-6sjmw\") pod \"kas-connection-checker-685895c7c9-4zl6m\" (UID: \"a4d58318-42ba-4a9e-9c74-43fe90baa4a7\") " pod="kube-system/kas-connection-checker-685895c7c9-4zl6m"
Oct 30 12:56:53.242746 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.242728    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qt8c9\" (UniqueName: \"kubernetes.io/projected/bd9e55af-64ad-4ffc-92c3-53b7bf654322-kube-api-access-qt8c9\") pod \"insights-operator-7c4986d449-7cfvt\" (UID: \"bd9e55af-64ad-4ffc-92c3-53b7bf654322\") " pod="openshift-insights/insights-operator-7c4986d449-7cfvt"
Oct 30 12:56:53.243067 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.243042    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rls7j\" (UniqueName: \"kubernetes.io/projected/85f4c399-0089-4399-86e2-069834f89286-kube-api-access-rls7j\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-plmkt\" (UID: \"85f4c399-0089-4399-86e2-069834f89286\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-plmkt"
Oct 30 12:56:53.243167 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.243117    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l8pqr\" (UniqueName: \"kubernetes.io/projected/b86e8e45-3c2d-46b6-9ade-d390b56e4a75-kube-api-access-l8pqr\") pod \"image-registry-69c87564f5-n6wjk\" (UID: \"b86e8e45-3c2d-46b6-9ade-d390b56e4a75\") " pod="openshift-image-registry/image-registry-69c87564f5-n6wjk"
Oct 30 12:56:53.243473 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.243457    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hfnqv\" (UniqueName: \"kubernetes.io/projected/27c55f84-4e9a-44de-94ea-02258d6bc747-kube-api-access-hfnqv\") pod \"service-ca-operator-544d749c8c-h42v2\" (UID: \"27c55f84-4e9a-44de-94ea-02258d6bc747\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-h42v2"
Oct 30 12:56:53.244599 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.244580    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qwmfv\" (UniqueName: \"kubernetes.io/projected/19b38c46-bfd4-4ed0-a764-f7b672f18cc6-kube-api-access-qwmfv\") pod \"cluster-monitoring-operator-884576c57-2sfj9\" (UID: \"19b38c46-bfd4-4ed0-a764-f7b672f18cc6\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-2sfj9"
Oct 30 12:56:53.245493 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.245474    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b86e8e45-3c2d-46b6-9ade-d390b56e4a75-bound-sa-token\") pod \"image-registry-69c87564f5-n6wjk\" (UID: \"b86e8e45-3c2d-46b6-9ade-d390b56e4a75\") " pod="openshift-image-registry/image-registry-69c87564f5-n6wjk"
Oct 30 12:56:53.259548 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.259531    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kas-connection-checker-685895c7c9-h7brb"
Oct 30 12:56:53.271463 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.271445    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-xtfpg"
Oct 30 12:56:53.281788 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.281763    2573 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-5bbfv\""
Oct 30 12:56:53.281788 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.281779    2573 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"ingress-canary-dockercfg-sdfnl\""
Oct 30 12:56:53.281788 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.281775    2573 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Oct 30 12:56:53.281958 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.281890    2573 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Oct 30 12:56:53.289020 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.289000    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-7c4986d449-7cfvt"
Oct 30 12:56:53.303375 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.303351    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kas-connection-checker-685895c7c9-4zl6m"
Oct 30 12:56:53.317095 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.317068    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jz42t\" (UniqueName: \"kubernetes.io/projected/23297e74-48f9-47b7-bb5a-80189bdbf2c4-kube-api-access-jz42t\") pod \"cluster-samples-operator-6ddb6c8b94-7dtcd\" (UID: \"23297e74-48f9-47b7-bb5a-80189bdbf2c4\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-7dtcd"
Oct 30 12:56:53.317242 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.317101    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/1c405568-cb88-4485-ae2c-3614cc31ced6-registry-certificates\") pod \"image-registry-77c6ff895d-vw6c6\" (UID: \"1c405568-cb88-4485-ae2c-3614cc31ced6\") " pod="openshift-image-registry/image-registry-77c6ff895d-vw6c6"
Oct 30 12:56:53.317242 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.317119    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1c405568-cb88-4485-ae2c-3614cc31ced6-bound-sa-token\") pod \"image-registry-77c6ff895d-vw6c6\" (UID: \"1c405568-cb88-4485-ae2c-3614cc31ced6\") " pod="openshift-image-registry/image-registry-77c6ff895d-vw6c6"
Oct 30 12:56:53.317242 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.317136    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-54kft\" (UniqueName: \"kubernetes.io/projected/1c405568-cb88-4485-ae2c-3614cc31ced6-kube-api-access-54kft\") pod \"image-registry-77c6ff895d-vw6c6\" (UID: \"1c405568-cb88-4485-ae2c-3614cc31ced6\") " pod="openshift-image-registry/image-registry-77c6ff895d-vw6c6"
Oct 30 12:56:53.317242 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.317175    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/23297e74-48f9-47b7-bb5a-80189bdbf2c4-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-7dtcd\" (UID: \"23297e74-48f9-47b7-bb5a-80189bdbf2c4\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-7dtcd"
Oct 30 12:56:53.317242 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.317192    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/d8930ee7-0ae0-48b5-8ce3-cb8820355cef-config\") pod \"console-operator-7cd56b8dbc-8fqfr\" (UID: \"d8930ee7-0ae0-48b5-8ce3-cb8820355cef\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-8fqfr"
Oct 30 12:56:53.317242 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.317214    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/e1f629e6-6a96-444f-8329-e71bb98c9029-stats-auth\") pod \"router-default-649bdc978f-lkpqz\" (UID: \"e1f629e6-6a96-444f-8329-e71bb98c9029\") " pod="openshift-ingress/router-default-649bdc978f-lkpqz"
Oct 30 12:56:53.317242 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.317232    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fcf99170-f5cf-4987-97bd-1c30b1b37a35-tmp-dir\") pod \"dns-default-g2ckh\" (UID: \"fcf99170-f5cf-4987-97bd-1c30b1b37a35\") " pod="openshift-dns/dns-default-g2ckh"
Oct 30 12:56:53.317700 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.317275    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/fcf99170-f5cf-4987-97bd-1c30b1b37a35-config-volume\") pod \"dns-default-g2ckh\" (UID: \"fcf99170-f5cf-4987-97bd-1c30b1b37a35\") " pod="openshift-dns/dns-default-g2ckh"
Oct 30 12:56:53.317700 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.317309    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gz9d6\" (UniqueName: \"kubernetes.io/projected/7ed65395-2cbd-46bf-81dc-fe307f3a9345-kube-api-access-gz9d6\") pod \"network-check-source-6f78d59f99-vdwgx\" (UID: \"7ed65395-2cbd-46bf-81dc-fe307f3a9345\") " pod="openshift-network-diagnostics/network-check-source-6f78d59f99-vdwgx"
Oct 30 12:56:53.317700 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.317326    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/4883c26d-0877-4870-b0ee-406f5bfa8cf4-nginx-conf\") pod \"networking-console-plugin-86c478bb88-vsqgb\" (UID: \"4883c26d-0877-4870-b0ee-406f5bfa8cf4\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-vsqgb"
Oct 30 12:56:53.317700 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.317348    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e1f629e6-6a96-444f-8329-e71bb98c9029-service-ca-bundle\") pod \"router-default-649bdc978f-lkpqz\" (UID: \"e1f629e6-6a96-444f-8329-e71bb98c9029\") " pod="openshift-ingress/router-default-649bdc978f-lkpqz"
Oct 30 12:56:53.320295 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.317368    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-config\" (UniqueName: \"kubernetes.io/configmap/23297e74-48f9-47b7-bb5a-80189bdbf2c4-samples-operator-config\") pod \"cluster-samples-operator-6ddb6c8b94-7dtcd\" (UID: \"23297e74-48f9-47b7-bb5a-80189bdbf2c4\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-7dtcd"
Oct 30 12:56:53.320295 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.319038    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/d8930ee7-0ae0-48b5-8ce3-cb8820355cef-config\") pod \"console-operator-7cd56b8dbc-8fqfr\" (UID: \"d8930ee7-0ae0-48b5-8ce3-cb8820355cef\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-8fqfr"
Oct 30 12:56:53.320590 ip-10-0-142-84 kubenswrapper[2573]: E1030 12:56:53.319670    2573 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Oct 30 12:56:53.320590 ip-10-0-142-84 kubenswrapper[2573]: E1030 12:56:53.319808    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e1f629e6-6a96-444f-8329-e71bb98c9029-service-ca-bundle podName:e1f629e6-6a96-444f-8329-e71bb98c9029 nodeName:}" failed. No retries permitted until 2026-10-30 12:56:53.819785663 +0000 UTC m=+31.219839022 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/e1f629e6-6a96-444f-8329-e71bb98c9029-service-ca-bundle") pod "router-default-649bdc978f-lkpqz" (UID: "e1f629e6-6a96-444f-8329-e71bb98c9029") : configmap references non-existent config key: service-ca.crt
Oct 30 12:56:53.320590 ip-10-0-142-84 kubenswrapper[2573]: E1030 12:56:53.320406    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/23297e74-48f9-47b7-bb5a-80189bdbf2c4-samples-operator-tls podName:23297e74-48f9-47b7-bb5a-80189bdbf2c4 nodeName:}" failed. No retries permitted until 2026-10-30 12:56:53.820388871 +0000 UTC m=+31.220442219 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/23297e74-48f9-47b7-bb5a-80189bdbf2c4-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-7dtcd" (UID: "23297e74-48f9-47b7-bb5a-80189bdbf2c4") : secret "samples-operator-tls" not found
Oct 30 12:56:53.320590 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.320425    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/d8930ee7-0ae0-48b5-8ce3-cb8820355cef-serving-cert\") pod \"console-operator-7cd56b8dbc-8fqfr\" (UID: \"d8930ee7-0ae0-48b5-8ce3-cb8820355cef\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-8fqfr"
Oct 30 12:56:53.320590 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.320468    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/98e7e1f3-2ca6-45de-a5d8-d27c22e257b1-cert\") pod \"ingress-canary-bdf5s\" (UID: \"98e7e1f3-2ca6-45de-a5d8-d27c22e257b1\") " pod="openshift-ingress-canary/ingress-canary-bdf5s"
Oct 30 12:56:53.321057 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.321018    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"samples-operator-config\" (UniqueName: \"kubernetes.io/configmap/23297e74-48f9-47b7-bb5a-80189bdbf2c4-samples-operator-config\") pod \"cluster-samples-operator-6ddb6c8b94-7dtcd\" (UID: \"23297e74-48f9-47b7-bb5a-80189bdbf2c4\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-7dtcd"
Oct 30 12:56:53.321143 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.321108    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hbsbr\" (UniqueName: \"kubernetes.io/projected/98e7e1f3-2ca6-45de-a5d8-d27c22e257b1-kube-api-access-hbsbr\") pod \"ingress-canary-bdf5s\" (UID: \"98e7e1f3-2ca6-45de-a5d8-d27c22e257b1\") " pod="openshift-ingress-canary/ingress-canary-bdf5s"
Oct 30 12:56:53.321232 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.321167    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/4883c26d-0877-4870-b0ee-406f5bfa8cf4-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-vsqgb\" (UID: \"4883c26d-0877-4870-b0ee-406f5bfa8cf4\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-vsqgb"
Oct 30 12:56:53.321232 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.321213    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e1f629e6-6a96-444f-8329-e71bb98c9029-metrics-certs\") pod \"router-default-649bdc978f-lkpqz\" (UID: \"e1f629e6-6a96-444f-8329-e71bb98c9029\") " pod="openshift-ingress/router-default-649bdc978f-lkpqz"
Oct 30 12:56:53.321395 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.321235    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1c405568-cb88-4485-ae2c-3614cc31ced6-registry-tls\") pod \"image-registry-77c6ff895d-vw6c6\" (UID: \"1c405568-cb88-4485-ae2c-3614cc31ced6\") " pod="openshift-image-registry/image-registry-77c6ff895d-vw6c6"
Oct 30 12:56:53.321395 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.321278    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/1c405568-cb88-4485-ae2c-3614cc31ced6-trusted-ca\") pod \"image-registry-77c6ff895d-vw6c6\" (UID: \"1c405568-cb88-4485-ae2c-3614cc31ced6\") " pod="openshift-image-registry/image-registry-77c6ff895d-vw6c6"
Oct 30 12:56:53.321395 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.321309    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/e1f629e6-6a96-444f-8329-e71bb98c9029-default-certificate\") pod \"router-default-649bdc978f-lkpqz\" (UID: \"e1f629e6-6a96-444f-8329-e71bb98c9029\") " pod="openshift-ingress/router-default-649bdc978f-lkpqz"
Oct 30 12:56:53.321395 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.321332    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d8930ee7-0ae0-48b5-8ce3-cb8820355cef-trusted-ca\") pod \"console-operator-7cd56b8dbc-8fqfr\" (UID: \"d8930ee7-0ae0-48b5-8ce3-cb8820355cef\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-8fqfr"
Oct 30 12:56:53.321395 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.321366    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tltj6\" (UniqueName: \"kubernetes.io/projected/d8930ee7-0ae0-48b5-8ce3-cb8820355cef-kube-api-access-tltj6\") pod \"console-operator-7cd56b8dbc-8fqfr\" (UID: \"d8930ee7-0ae0-48b5-8ce3-cb8820355cef\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-8fqfr"
Oct 30 12:56:53.321790 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.321420    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/fcf99170-f5cf-4987-97bd-1c30b1b37a35-metrics-tls\") pod \"dns-default-g2ckh\" (UID: \"fcf99170-f5cf-4987-97bd-1c30b1b37a35\") " pod="openshift-dns/dns-default-g2ckh"
Oct 30 12:56:53.321790 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.321442    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/1c405568-cb88-4485-ae2c-3614cc31ced6-installation-pull-secrets\") pod \"image-registry-77c6ff895d-vw6c6\" (UID: \"1c405568-cb88-4485-ae2c-3614cc31ced6\") " pod="openshift-image-registry/image-registry-77c6ff895d-vw6c6"
Oct 30 12:56:53.321790 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.321485    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-phkm5\" (UniqueName: \"kubernetes.io/projected/e1f629e6-6a96-444f-8329-e71bb98c9029-kube-api-access-phkm5\") pod \"router-default-649bdc978f-lkpqz\" (UID: \"e1f629e6-6a96-444f-8329-e71bb98c9029\") " pod="openshift-ingress/router-default-649bdc978f-lkpqz"
Oct 30 12:56:53.321790 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.321515    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/1c405568-cb88-4485-ae2c-3614cc31ced6-image-registry-private-configuration\") pod \"image-registry-77c6ff895d-vw6c6\" (UID: \"1c405568-cb88-4485-ae2c-3614cc31ced6\") " pod="openshift-image-registry/image-registry-77c6ff895d-vw6c6"
Oct 30 12:56:53.321790 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.321542    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/1c405568-cb88-4485-ae2c-3614cc31ced6-ca-trust-extracted\") pod \"image-registry-77c6ff895d-vw6c6\" (UID: \"1c405568-cb88-4485-ae2c-3614cc31ced6\") " pod="openshift-image-registry/image-registry-77c6ff895d-vw6c6"
Oct 30 12:56:53.321790 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.321567    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qxmjj\" (UniqueName: \"kubernetes.io/projected/fcf99170-f5cf-4987-97bd-1c30b1b37a35-kube-api-access-qxmjj\") pod \"dns-default-g2ckh\" (UID: \"fcf99170-f5cf-4987-97bd-1c30b1b37a35\") " pod="openshift-dns/dns-default-g2ckh"
Oct 30 12:56:53.321790 ip-10-0-142-84 kubenswrapper[2573]: E1030 12:56:53.321736    2573 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Oct 30 12:56:53.321790 ip-10-0-142-84 kubenswrapper[2573]: E1030 12:56:53.321751    2573 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-77c6ff895d-vw6c6: secret "image-registry-tls" not found
Oct 30 12:56:53.321790 ip-10-0-142-84 kubenswrapper[2573]: E1030 12:56:53.321795    2573 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Oct 30 12:56:53.323084 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.323057    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/4883c26d-0877-4870-b0ee-406f5bfa8cf4-nginx-conf\") pod \"networking-console-plugin-86c478bb88-vsqgb\" (UID: \"4883c26d-0877-4870-b0ee-406f5bfa8cf4\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-vsqgb"
Oct 30 12:56:53.323234 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.323214    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/1c405568-cb88-4485-ae2c-3614cc31ced6-registry-certificates\") pod \"image-registry-77c6ff895d-vw6c6\" (UID: \"1c405568-cb88-4485-ae2c-3614cc31ced6\") " pod="openshift-image-registry/image-registry-77c6ff895d-vw6c6"
Oct 30 12:56:53.328753 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.328251    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d8930ee7-0ae0-48b5-8ce3-cb8820355cef-trusted-ca\") pod \"console-operator-7cd56b8dbc-8fqfr\" (UID: \"d8930ee7-0ae0-48b5-8ce3-cb8820355cef\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-8fqfr"
Oct 30 12:56:53.343074 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.335826    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/e1f629e6-6a96-444f-8329-e71bb98c9029-stats-auth\") pod \"router-default-649bdc978f-lkpqz\" (UID: \"e1f629e6-6a96-444f-8329-e71bb98c9029\") " pod="openshift-ingress/router-default-649bdc978f-lkpqz"
Oct 30 12:56:53.343074 ip-10-0-142-84 kubenswrapper[2573]: E1030 12:56:53.339206    2573 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Oct 30 12:56:53.343074 ip-10-0-142-84 kubenswrapper[2573]: E1030 12:56:53.339289    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4883c26d-0877-4870-b0ee-406f5bfa8cf4-networking-console-plugin-cert podName:4883c26d-0877-4870-b0ee-406f5bfa8cf4 nodeName:}" failed. No retries permitted until 2026-10-30 12:56:53.839249307 +0000 UTC m=+31.239302649 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/4883c26d-0877-4870-b0ee-406f5bfa8cf4-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-vsqgb" (UID: "4883c26d-0877-4870-b0ee-406f5bfa8cf4") : secret "networking-console-plugin-cert" not found
Oct 30 12:56:53.343074 ip-10-0-142-84 kubenswrapper[2573]: E1030 12:56:53.339586    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1c405568-cb88-4485-ae2c-3614cc31ced6-registry-tls podName:1c405568-cb88-4485-ae2c-3614cc31ced6 nodeName:}" failed. No retries permitted until 2026-10-30 12:56:53.839570071 +0000 UTC m=+31.239623409 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/1c405568-cb88-4485-ae2c-3614cc31ced6-registry-tls") pod "image-registry-77c6ff895d-vw6c6" (UID: "1c405568-cb88-4485-ae2c-3614cc31ced6") : secret "image-registry-tls" not found
Oct 30 12:56:53.343074 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.339779    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/e1f629e6-6a96-444f-8329-e71bb98c9029-default-certificate\") pod \"router-default-649bdc978f-lkpqz\" (UID: \"e1f629e6-6a96-444f-8329-e71bb98c9029\") " pod="openshift-ingress/router-default-649bdc978f-lkpqz"
Oct 30 12:56:53.343074 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.340170    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/1c405568-cb88-4485-ae2c-3614cc31ced6-ca-trust-extracted\") pod \"image-registry-77c6ff895d-vw6c6\" (UID: \"1c405568-cb88-4485-ae2c-3614cc31ced6\") " pod="openshift-image-registry/image-registry-77c6ff895d-vw6c6"
Oct 30 12:56:53.343074 ip-10-0-142-84 kubenswrapper[2573]: E1030 12:56:53.341469    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e1f629e6-6a96-444f-8329-e71bb98c9029-metrics-certs podName:e1f629e6-6a96-444f-8329-e71bb98c9029 nodeName:}" failed. No retries permitted until 2026-10-30 12:56:53.841435757 +0000 UTC m=+31.241489103 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e1f629e6-6a96-444f-8329-e71bb98c9029-metrics-certs") pod "router-default-649bdc978f-lkpqz" (UID: "e1f629e6-6a96-444f-8329-e71bb98c9029") : secret "router-metrics-certs-default" not found
Oct 30 12:56:53.347290 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.344522    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/1c405568-cb88-4485-ae2c-3614cc31ced6-image-registry-private-configuration\") pod \"image-registry-77c6ff895d-vw6c6\" (UID: \"1c405568-cb88-4485-ae2c-3614cc31ced6\") " pod="openshift-image-registry/image-registry-77c6ff895d-vw6c6"
Oct 30 12:56:53.351467 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.350880    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/d8930ee7-0ae0-48b5-8ce3-cb8820355cef-serving-cert\") pod \"console-operator-7cd56b8dbc-8fqfr\" (UID: \"d8930ee7-0ae0-48b5-8ce3-cb8820355cef\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-8fqfr"
Oct 30 12:56:53.351467 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.351129    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-h42v2"
Oct 30 12:56:53.351467 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.351418    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/1c405568-cb88-4485-ae2c-3614cc31ced6-trusted-ca\") pod \"image-registry-77c6ff895d-vw6c6\" (UID: \"1c405568-cb88-4485-ae2c-3614cc31ced6\") " pod="openshift-image-registry/image-registry-77c6ff895d-vw6c6"
Oct 30 12:56:53.364790 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.364760    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/1c405568-cb88-4485-ae2c-3614cc31ced6-installation-pull-secrets\") pod \"image-registry-77c6ff895d-vw6c6\" (UID: \"1c405568-cb88-4485-ae2c-3614cc31ced6\") " pod="openshift-image-registry/image-registry-77c6ff895d-vw6c6"
Oct 30 12:56:53.365342 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.365320    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-plmkt"
Oct 30 12:56:53.367790 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.367481    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jz42t\" (UniqueName: \"kubernetes.io/projected/23297e74-48f9-47b7-bb5a-80189bdbf2c4-kube-api-access-jz42t\") pod \"cluster-samples-operator-6ddb6c8b94-7dtcd\" (UID: \"23297e74-48f9-47b7-bb5a-80189bdbf2c4\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-7dtcd"
Oct 30 12:56:53.367790 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.367755    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-54kft\" (UniqueName: \"kubernetes.io/projected/1c405568-cb88-4485-ae2c-3614cc31ced6-kube-api-access-54kft\") pod \"image-registry-77c6ff895d-vw6c6\" (UID: \"1c405568-cb88-4485-ae2c-3614cc31ced6\") " pod="openshift-image-registry/image-registry-77c6ff895d-vw6c6"
Oct 30 12:56:53.372068 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.371849    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tltj6\" (UniqueName: \"kubernetes.io/projected/d8930ee7-0ae0-48b5-8ce3-cb8820355cef-kube-api-access-tltj6\") pod \"console-operator-7cd56b8dbc-8fqfr\" (UID: \"d8930ee7-0ae0-48b5-8ce3-cb8820355cef\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-8fqfr"
Oct 30 12:56:53.377791 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.376314    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1c405568-cb88-4485-ae2c-3614cc31ced6-bound-sa-token\") pod \"image-registry-77c6ff895d-vw6c6\" (UID: \"1c405568-cb88-4485-ae2c-3614cc31ced6\") " pod="openshift-image-registry/image-registry-77c6ff895d-vw6c6"
Oct 30 12:56:53.377791 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.377688    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-phkm5\" (UniqueName: \"kubernetes.io/projected/e1f629e6-6a96-444f-8329-e71bb98c9029-kube-api-access-phkm5\") pod \"router-default-649bdc978f-lkpqz\" (UID: \"e1f629e6-6a96-444f-8329-e71bb98c9029\") " pod="openshift-ingress/router-default-649bdc978f-lkpqz"
Oct 30 12:56:53.440287 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.439382    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qxmjj\" (UniqueName: \"kubernetes.io/projected/fcf99170-f5cf-4987-97bd-1c30b1b37a35-kube-api-access-qxmjj\") pod \"dns-default-g2ckh\" (UID: \"fcf99170-f5cf-4987-97bd-1c30b1b37a35\") " pod="openshift-dns/dns-default-g2ckh"
Oct 30 12:56:53.440287 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.439476    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fcf99170-f5cf-4987-97bd-1c30b1b37a35-tmp-dir\") pod \"dns-default-g2ckh\" (UID: \"fcf99170-f5cf-4987-97bd-1c30b1b37a35\") " pod="openshift-dns/dns-default-g2ckh"
Oct 30 12:56:53.440287 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.439518    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/fcf99170-f5cf-4987-97bd-1c30b1b37a35-config-volume\") pod \"dns-default-g2ckh\" (UID: \"fcf99170-f5cf-4987-97bd-1c30b1b37a35\") " pod="openshift-dns/dns-default-g2ckh"
Oct 30 12:56:53.440287 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.439556    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gz9d6\" (UniqueName: \"kubernetes.io/projected/7ed65395-2cbd-46bf-81dc-fe307f3a9345-kube-api-access-gz9d6\") pod \"network-check-source-6f78d59f99-vdwgx\" (UID: \"7ed65395-2cbd-46bf-81dc-fe307f3a9345\") " pod="openshift-network-diagnostics/network-check-source-6f78d59f99-vdwgx"
Oct 30 12:56:53.440287 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.439613    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/98e7e1f3-2ca6-45de-a5d8-d27c22e257b1-cert\") pod \"ingress-canary-bdf5s\" (UID: \"98e7e1f3-2ca6-45de-a5d8-d27c22e257b1\") " pod="openshift-ingress-canary/ingress-canary-bdf5s"
Oct 30 12:56:53.440287 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.439654    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hbsbr\" (UniqueName: \"kubernetes.io/projected/98e7e1f3-2ca6-45de-a5d8-d27c22e257b1-kube-api-access-hbsbr\") pod \"ingress-canary-bdf5s\" (UID: \"98e7e1f3-2ca6-45de-a5d8-d27c22e257b1\") " pod="openshift-ingress-canary/ingress-canary-bdf5s"
Oct 30 12:56:53.440287 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.439786    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/fcf99170-f5cf-4987-97bd-1c30b1b37a35-metrics-tls\") pod \"dns-default-g2ckh\" (UID: \"fcf99170-f5cf-4987-97bd-1c30b1b37a35\") " pod="openshift-dns/dns-default-g2ckh"
Oct 30 12:56:53.440287 ip-10-0-142-84 kubenswrapper[2573]: E1030 12:56:53.439954    2573 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Oct 30 12:56:53.440287 ip-10-0-142-84 kubenswrapper[2573]: E1030 12:56:53.440009    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fcf99170-f5cf-4987-97bd-1c30b1b37a35-metrics-tls podName:fcf99170-f5cf-4987-97bd-1c30b1b37a35 nodeName:}" failed. No retries permitted until 2026-10-30 12:56:53.939992283 +0000 UTC m=+31.340045638 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/fcf99170-f5cf-4987-97bd-1c30b1b37a35-metrics-tls") pod "dns-default-g2ckh" (UID: "fcf99170-f5cf-4987-97bd-1c30b1b37a35") : secret "dns-default-metrics-tls" not found
Oct 30 12:56:53.440287 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.440102    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fcf99170-f5cf-4987-97bd-1c30b1b37a35-tmp-dir\") pod \"dns-default-g2ckh\" (UID: \"fcf99170-f5cf-4987-97bd-1c30b1b37a35\") " pod="openshift-dns/dns-default-g2ckh"
Oct 30 12:56:53.440287 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.440157    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/fcf99170-f5cf-4987-97bd-1c30b1b37a35-config-volume\") pod \"dns-default-g2ckh\" (UID: \"fcf99170-f5cf-4987-97bd-1c30b1b37a35\") " pod="openshift-dns/dns-default-g2ckh"
Oct 30 12:56:53.444516 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.444491    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-7cd56b8dbc-8fqfr"
Oct 30 12:56:53.447085 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.446613    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/98e7e1f3-2ca6-45de-a5d8-d27c22e257b1-cert\") pod \"ingress-canary-bdf5s\" (UID: \"98e7e1f3-2ca6-45de-a5d8-d27c22e257b1\") " pod="openshift-ingress-canary/ingress-canary-bdf5s"
Oct 30 12:56:53.464188 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.464167    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qxmjj\" (UniqueName: \"kubernetes.io/projected/fcf99170-f5cf-4987-97bd-1c30b1b37a35-kube-api-access-qxmjj\") pod \"dns-default-g2ckh\" (UID: \"fcf99170-f5cf-4987-97bd-1c30b1b37a35\") " pod="openshift-dns/dns-default-g2ckh"
Oct 30 12:56:53.464519 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.464496    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gz9d6\" (UniqueName: \"kubernetes.io/projected/7ed65395-2cbd-46bf-81dc-fe307f3a9345-kube-api-access-gz9d6\") pod \"network-check-source-6f78d59f99-vdwgx\" (UID: \"7ed65395-2cbd-46bf-81dc-fe307f3a9345\") " pod="openshift-network-diagnostics/network-check-source-6f78d59f99-vdwgx"
Oct 30 12:56:53.468980 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.468958    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hbsbr\" (UniqueName: \"kubernetes.io/projected/98e7e1f3-2ca6-45de-a5d8-d27c22e257b1-kube-api-access-hbsbr\") pod \"ingress-canary-bdf5s\" (UID: \"98e7e1f3-2ca6-45de-a5d8-d27c22e257b1\") " pod="openshift-ingress-canary/ingress-canary-bdf5s"
Oct 30 12:56:53.497688 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.497668    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-6f78d59f99-vdwgx"
Oct 30 12:56:53.501824 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.501806    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-bdf5s"
Oct 30 12:56:53.589121 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.589002    2573 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-k4v7j"]
Oct 30 12:56:53.591369 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:53.591324    2573 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0905782c_26ad_4dc8_93cf_d6f211ea7af5.slice/crio-d8be95465e29616c512fecb557b5599168658616bffa156020d86ab0a30942f4 WatchSource:0}: Error finding container d8be95465e29616c512fecb557b5599168658616bffa156020d86ab0a30942f4: Status 404 returned error can't find the container with id d8be95465e29616c512fecb557b5599168658616bffa156020d86ab0a30942f4
Oct 30 12:56:53.595067 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.593670    2573 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-h7brb"]
Oct 30 12:56:53.595067 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.594385    2573 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-xtfpg"]
Oct 30 12:56:53.639440 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.639411    2573 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-7c4986d449-7cfvt"]
Oct 30 12:56:53.652188 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.652084    2573 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-4zl6m"]
Oct 30 12:56:53.659536 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:53.659498    2573 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda4d58318_42ba_4a9e_9c74_43fe90baa4a7.slice/crio-c501eda152de5cd4a5dedb5febbd60de386dfd9fd8cc677a2ebf5afebd205cad WatchSource:0}: Error finding container c501eda152de5cd4a5dedb5febbd60de386dfd9fd8cc677a2ebf5afebd205cad: Status 404 returned error can't find the container with id c501eda152de5cd4a5dedb5febbd60de386dfd9fd8cc677a2ebf5afebd205cad
Oct 30 12:56:53.710456 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.710382    2573 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-544d749c8c-h42v2"]
Oct 30 12:56:53.715170 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.715104    2573 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-plmkt"]
Oct 30 12:56:53.715952 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:53.715922    2573 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod27c55f84_4e9a_44de_94ea_02258d6bc747.slice/crio-15f0ea4b68ee48208edbca8d812819c1cfee317092c13305aa22c767e5ce6480 WatchSource:0}: Error finding container 15f0ea4b68ee48208edbca8d812819c1cfee317092c13305aa22c767e5ce6480: Status 404 returned error can't find the container with id 15f0ea4b68ee48208edbca8d812819c1cfee317092c13305aa22c767e5ce6480
Oct 30 12:56:53.744057 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.743696    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b86e8e45-3c2d-46b6-9ade-d390b56e4a75-registry-tls\") pod \"image-registry-69c87564f5-n6wjk\" (UID: \"b86e8e45-3c2d-46b6-9ade-d390b56e4a75\") " pod="openshift-image-registry/image-registry-69c87564f5-n6wjk"
Oct 30 12:56:53.744057 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.743789    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/19b38c46-bfd4-4ed0-a764-f7b672f18cc6-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-2sfj9\" (UID: \"19b38c46-bfd4-4ed0-a764-f7b672f18cc6\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-2sfj9"
Oct 30 12:56:53.744057 ip-10-0-142-84 kubenswrapper[2573]: E1030 12:56:53.743927    2573 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Oct 30 12:56:53.744057 ip-10-0-142-84 kubenswrapper[2573]: E1030 12:56:53.743967    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/19b38c46-bfd4-4ed0-a764-f7b672f18cc6-cluster-monitoring-operator-tls podName:19b38c46-bfd4-4ed0-a764-f7b672f18cc6 nodeName:}" failed. No retries permitted until 2026-10-30 12:56:54.74395514 +0000 UTC m=+32.144008491 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/19b38c46-bfd4-4ed0-a764-f7b672f18cc6-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-2sfj9" (UID: "19b38c46-bfd4-4ed0-a764-f7b672f18cc6") : secret "cluster-monitoring-operator-tls" not found
Oct 30 12:56:53.744057 ip-10-0-142-84 kubenswrapper[2573]: E1030 12:56:53.744014    2573 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Oct 30 12:56:53.744057 ip-10-0-142-84 kubenswrapper[2573]: E1030 12:56:53.744021    2573 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-69c87564f5-n6wjk: secret "image-registry-tls" not found
Oct 30 12:56:53.744057 ip-10-0-142-84 kubenswrapper[2573]: E1030 12:56:53.744041    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b86e8e45-3c2d-46b6-9ade-d390b56e4a75-registry-tls podName:b86e8e45-3c2d-46b6-9ade-d390b56e4a75 nodeName:}" failed. No retries permitted until 2026-10-30 12:56:54.744034866 +0000 UTC m=+32.144088217 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/b86e8e45-3c2d-46b6-9ade-d390b56e4a75-registry-tls") pod "image-registry-69c87564f5-n6wjk" (UID: "b86e8e45-3c2d-46b6-9ade-d390b56e4a75") : secret "image-registry-tls" not found
Oct 30 12:56:53.787569 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.787514    2573 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-7cd56b8dbc-8fqfr"]
Oct 30 12:56:53.797245 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:53.797222    2573 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd8930ee7_0ae0_48b5_8ce3_cb8820355cef.slice/crio-ef64188e81684686678c54e0cb2634c8321a9f8e65abacee44523832bc1b9c45 WatchSource:0}: Error finding container ef64188e81684686678c54e0cb2634c8321a9f8e65abacee44523832bc1b9c45: Status 404 returned error can't find the container with id ef64188e81684686678c54e0cb2634c8321a9f8e65abacee44523832bc1b9c45
Oct 30 12:56:53.844578 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.844531    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/23297e74-48f9-47b7-bb5a-80189bdbf2c4-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-7dtcd\" (UID: \"23297e74-48f9-47b7-bb5a-80189bdbf2c4\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-7dtcd"
Oct 30 12:56:53.844690 ip-10-0-142-84 kubenswrapper[2573]: E1030 12:56:53.844670    2573 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Oct 30 12:56:53.844743 ip-10-0-142-84 kubenswrapper[2573]: E1030 12:56:53.844729    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/23297e74-48f9-47b7-bb5a-80189bdbf2c4-samples-operator-tls podName:23297e74-48f9-47b7-bb5a-80189bdbf2c4 nodeName:}" failed. No retries permitted until 2026-10-30 12:56:54.844709519 +0000 UTC m=+32.244762860 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/23297e74-48f9-47b7-bb5a-80189bdbf2c4-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-7dtcd" (UID: "23297e74-48f9-47b7-bb5a-80189bdbf2c4") : secret "samples-operator-tls" not found
Oct 30 12:56:53.844800 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.844679    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e1f629e6-6a96-444f-8329-e71bb98c9029-service-ca-bundle\") pod \"router-default-649bdc978f-lkpqz\" (UID: \"e1f629e6-6a96-444f-8329-e71bb98c9029\") " pod="openshift-ingress/router-default-649bdc978f-lkpqz"
Oct 30 12:56:53.844800 ip-10-0-142-84 kubenswrapper[2573]: E1030 12:56:53.844792    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e1f629e6-6a96-444f-8329-e71bb98c9029-service-ca-bundle podName:e1f629e6-6a96-444f-8329-e71bb98c9029 nodeName:}" failed. No retries permitted until 2026-10-30 12:56:54.844780079 +0000 UTC m=+32.244833451 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/e1f629e6-6a96-444f-8329-e71bb98c9029-service-ca-bundle") pod "router-default-649bdc978f-lkpqz" (UID: "e1f629e6-6a96-444f-8329-e71bb98c9029") : configmap references non-existent config key: service-ca.crt
Oct 30 12:56:53.844875 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.844862    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/4883c26d-0877-4870-b0ee-406f5bfa8cf4-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-vsqgb\" (UID: \"4883c26d-0877-4870-b0ee-406f5bfa8cf4\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-vsqgb"
Oct 30 12:56:53.844910 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.844887    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e1f629e6-6a96-444f-8329-e71bb98c9029-metrics-certs\") pod \"router-default-649bdc978f-lkpqz\" (UID: \"e1f629e6-6a96-444f-8329-e71bb98c9029\") " pod="openshift-ingress/router-default-649bdc978f-lkpqz"
Oct 30 12:56:53.844947 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.844909    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1c405568-cb88-4485-ae2c-3614cc31ced6-registry-tls\") pod \"image-registry-77c6ff895d-vw6c6\" (UID: \"1c405568-cb88-4485-ae2c-3614cc31ced6\") " pod="openshift-image-registry/image-registry-77c6ff895d-vw6c6"
Oct 30 12:56:53.844998 ip-10-0-142-84 kubenswrapper[2573]: E1030 12:56:53.844982    2573 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Oct 30 12:56:53.845047 ip-10-0-142-84 kubenswrapper[2573]: E1030 12:56:53.845019    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4883c26d-0877-4870-b0ee-406f5bfa8cf4-networking-console-plugin-cert podName:4883c26d-0877-4870-b0ee-406f5bfa8cf4 nodeName:}" failed. No retries permitted until 2026-10-30 12:56:54.845009808 +0000 UTC m=+32.245063150 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/4883c26d-0877-4870-b0ee-406f5bfa8cf4-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-vsqgb" (UID: "4883c26d-0877-4870-b0ee-406f5bfa8cf4") : secret "networking-console-plugin-cert" not found
Oct 30 12:56:53.845047 ip-10-0-142-84 kubenswrapper[2573]: E1030 12:56:53.845021    2573 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Oct 30 12:56:53.845047 ip-10-0-142-84 kubenswrapper[2573]: E1030 12:56:53.844982    2573 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Oct 30 12:56:53.845047 ip-10-0-142-84 kubenswrapper[2573]: E1030 12:56:53.845031    2573 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-77c6ff895d-vw6c6: secret "image-registry-tls" not found
Oct 30 12:56:53.845047 ip-10-0-142-84 kubenswrapper[2573]: E1030 12:56:53.845047    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e1f629e6-6a96-444f-8329-e71bb98c9029-metrics-certs podName:e1f629e6-6a96-444f-8329-e71bb98c9029 nodeName:}" failed. No retries permitted until 2026-10-30 12:56:54.845039663 +0000 UTC m=+32.245093057 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e1f629e6-6a96-444f-8329-e71bb98c9029-metrics-certs") pod "router-default-649bdc978f-lkpqz" (UID: "e1f629e6-6a96-444f-8329-e71bb98c9029") : secret "router-metrics-certs-default" not found
Oct 30 12:56:53.845292 ip-10-0-142-84 kubenswrapper[2573]: E1030 12:56:53.845062    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1c405568-cb88-4485-ae2c-3614cc31ced6-registry-tls podName:1c405568-cb88-4485-ae2c-3614cc31ced6 nodeName:}" failed. No retries permitted until 2026-10-30 12:56:54.845053962 +0000 UTC m=+32.245107299 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/1c405568-cb88-4485-ae2c-3614cc31ced6-registry-tls") pod "image-registry-77c6ff895d-vw6c6" (UID: "1c405568-cb88-4485-ae2c-3614cc31ced6") : secret "image-registry-tls" not found
Oct 30 12:56:53.862840 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.862809    2573 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-bdf5s"]
Oct 30 12:56:53.865641 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.865622    2573 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-6f78d59f99-vdwgx"]
Oct 30 12:56:53.867324 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:53.867302    2573 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod98e7e1f3_2ca6_45de_a5d8_d27c22e257b1.slice/crio-e463ce667e1fda351b2fee3fbb5b5a5194516d6c0afc229100c0f8501e19fe57 WatchSource:0}: Error finding container e463ce667e1fda351b2fee3fbb5b5a5194516d6c0afc229100c0f8501e19fe57: Status 404 returned error can't find the container with id e463ce667e1fda351b2fee3fbb5b5a5194516d6c0afc229100c0f8501e19fe57
Oct 30 12:56:53.870107 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:53.870089    2573 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7ed65395_2cbd_46bf_81dc_fe307f3a9345.slice/crio-f64856dcc52a80712efd8546500b0aa1dcf1f82708d8fc6b33eca1a2baa4320c WatchSource:0}: Error finding container f64856dcc52a80712efd8546500b0aa1dcf1f82708d8fc6b33eca1a2baa4320c: Status 404 returned error can't find the container with id f64856dcc52a80712efd8546500b0aa1dcf1f82708d8fc6b33eca1a2baa4320c
Oct 30 12:56:53.946009 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:53.945981    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/fcf99170-f5cf-4987-97bd-1c30b1b37a35-metrics-tls\") pod \"dns-default-g2ckh\" (UID: \"fcf99170-f5cf-4987-97bd-1c30b1b37a35\") " pod="openshift-dns/dns-default-g2ckh"
Oct 30 12:56:53.946110 ip-10-0-142-84 kubenswrapper[2573]: E1030 12:56:53.946080    2573 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Oct 30 12:56:53.946143 ip-10-0-142-84 kubenswrapper[2573]: E1030 12:56:53.946120    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fcf99170-f5cf-4987-97bd-1c30b1b37a35-metrics-tls podName:fcf99170-f5cf-4987-97bd-1c30b1b37a35 nodeName:}" failed. No retries permitted until 2026-10-30 12:56:54.946109103 +0000 UTC m=+32.346162441 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/fcf99170-f5cf-4987-97bd-1c30b1b37a35-metrics-tls") pod "dns-default-g2ckh" (UID: "fcf99170-f5cf-4987-97bd-1c30b1b37a35") : secret "dns-default-metrics-tls" not found
Oct 30 12:56:54.053327 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:54.053299    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p7tv6"
Oct 30 12:56:54.053517 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:54.053487    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rpdcd"
Oct 30 12:56:54.067896 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:54.067876    2573 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Oct 30 12:56:54.068148 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:54.068074    2573 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-sqtw6\""
Oct 30 12:56:54.068148 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:54.068089    2573 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-v4tv9\""
Oct 30 12:56:54.068148 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:54.068125    2573 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Oct 30 12:56:54.426982 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:54.426860    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-h42v2" event={"ID":"27c55f84-4e9a-44de-94ea-02258d6bc747","Type":"ContainerStarted","Data":"15f0ea4b68ee48208edbca8d812819c1cfee317092c13305aa22c767e5ce6480"}
Oct 30 12:56:54.431712 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:54.431658    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kas-connection-checker-685895c7c9-4zl6m" event={"ID":"a4d58318-42ba-4a9e-9c74-43fe90baa4a7","Type":"ContainerStarted","Data":"ab40e1c5df9f904b88ae7873da596452c2eb82c295c7d2c5cfae609470f88378"}
Oct 30 12:56:54.431948 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:54.431713    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kas-connection-checker-685895c7c9-4zl6m" event={"ID":"a4d58318-42ba-4a9e-9c74-43fe90baa4a7","Type":"ContainerStarted","Data":"c501eda152de5cd4a5dedb5febbd60de386dfd9fd8cc677a2ebf5afebd205cad"}
Oct 30 12:56:54.434527 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:54.434447    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-xtfpg" event={"ID":"f07abb3f-8900-4fa5-8d35-c7ab1d35582d","Type":"ContainerStarted","Data":"08239c831d96bfc6cf0bec4a44ccf8911e596dd61d29ac6eacc10da29b2e6b5e"}
Oct 30 12:56:54.436551 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:54.436487    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-bdf5s" event={"ID":"98e7e1f3-2ca6-45de-a5d8-d27c22e257b1","Type":"ContainerStarted","Data":"e463ce667e1fda351b2fee3fbb5b5a5194516d6c0afc229100c0f8501e19fe57"}
Oct 30 12:56:54.437811 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:54.437750    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-7c4986d449-7cfvt" event={"ID":"bd9e55af-64ad-4ffc-92c3-53b7bf654322","Type":"ContainerStarted","Data":"91591585f74679f5a203056eee04a9ff065c4244679fef41594168eba3ad7111"}
Oct 30 12:56:54.439873 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:54.439443    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kas-connection-checker-685895c7c9-h7brb" event={"ID":"301eebcb-3d48-4eef-a0de-a1173e23ad65","Type":"ContainerStarted","Data":"e794c11fbc8e35160795b43048e93aee324fbcfdf8597774880ff735b9bdcfa7"}
Oct 30 12:56:54.439873 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:54.439477    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kas-connection-checker-685895c7c9-h7brb" event={"ID":"301eebcb-3d48-4eef-a0de-a1173e23ad65","Type":"ContainerStarted","Data":"59fa8cb30606c434c1df30aa0f941a6c6d80de50607501c039b61cfc0a8f5759"}
Oct 30 12:56:54.444005 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:54.443909    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-6f78d59f99-vdwgx" event={"ID":"7ed65395-2cbd-46bf-81dc-fe307f3a9345","Type":"ContainerStarted","Data":"f64856dcc52a80712efd8546500b0aa1dcf1f82708d8fc6b33eca1a2baa4320c"}
Oct 30 12:56:54.448604 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:54.448063    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kas-connection-checker-685895c7c9-k4v7j" event={"ID":"0905782c-26ad-4dc8-93cf-d6f211ea7af5","Type":"ContainerStarted","Data":"31d1dc82a74220dcba7f4c1296eb6b8ae9893b19236d616c170d6110d9e5f7d4"}
Oct 30 12:56:54.448604 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:54.448104    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kas-connection-checker-685895c7c9-k4v7j" event={"ID":"0905782c-26ad-4dc8-93cf-d6f211ea7af5","Type":"ContainerStarted","Data":"d8be95465e29616c512fecb557b5599168658616bffa156020d86ab0a30942f4"}
Oct 30 12:56:54.456081 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:54.456051    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7cd56b8dbc-8fqfr" event={"ID":"d8930ee7-0ae0-48b5-8ce3-cb8820355cef","Type":"ContainerStarted","Data":"ef64188e81684686678c54e0cb2634c8321a9f8e65abacee44523832bc1b9c45"}
Oct 30 12:56:54.461386 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:54.461348    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-plmkt" event={"ID":"85f4c399-0089-4399-86e2-069834f89286","Type":"ContainerStarted","Data":"9387d634b449ee1cf65146c83f1fd3d16ba72de247544e9952f79af081174b7f"}
Oct 30 12:56:54.472032 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:54.470790    2573 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/kas-connection-checker-685895c7c9-4zl6m" podStartSLOduration=377.47077585 podStartE2EDuration="6m17.47077585s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 12:50:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 12:56:54.470094575 +0000 UTC m=+31.870147958" watchObservedRunningTime="2026-10-30 12:56:54.47077585 +0000 UTC m=+31.870829211"
Oct 30 12:56:54.516551 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:54.513955    2573 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/kas-connection-checker-685895c7c9-k4v7j" podStartSLOduration=377.513936685 podStartE2EDuration="6m17.513936685s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 12:50:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 12:56:54.513010461 +0000 UTC m=+31.913063823" watchObservedRunningTime="2026-10-30 12:56:54.513936685 +0000 UTC m=+31.913990047"
Oct 30 12:56:54.521291 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:54.520824    2573 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-zth9k"
Oct 30 12:56:54.521291 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:54.520945    2573 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Oct 30 12:56:54.521921 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:54.521820    2573 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-zth9k"
Oct 30 12:56:54.594425 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:54.592702    2573 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/kas-connection-checker-685895c7c9-h7brb" podStartSLOduration=377.592685928 podStartE2EDuration="6m17.592685928s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 12:50:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 12:56:54.554451227 +0000 UTC m=+31.954504588" watchObservedRunningTime="2026-10-30 12:56:54.592685928 +0000 UTC m=+31.992739286"
Oct 30 12:56:54.755952 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:54.754999    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/19b38c46-bfd4-4ed0-a764-f7b672f18cc6-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-2sfj9\" (UID: \"19b38c46-bfd4-4ed0-a764-f7b672f18cc6\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-2sfj9"
Oct 30 12:56:54.755952 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:54.755110    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b86e8e45-3c2d-46b6-9ade-d390b56e4a75-registry-tls\") pod \"image-registry-69c87564f5-n6wjk\" (UID: \"b86e8e45-3c2d-46b6-9ade-d390b56e4a75\") " pod="openshift-image-registry/image-registry-69c87564f5-n6wjk"
Oct 30 12:56:54.755952 ip-10-0-142-84 kubenswrapper[2573]: E1030 12:56:54.755326    2573 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Oct 30 12:56:54.755952 ip-10-0-142-84 kubenswrapper[2573]: E1030 12:56:54.755342    2573 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-69c87564f5-n6wjk: secret "image-registry-tls" not found
Oct 30 12:56:54.755952 ip-10-0-142-84 kubenswrapper[2573]: E1030 12:56:54.755395    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b86e8e45-3c2d-46b6-9ade-d390b56e4a75-registry-tls podName:b86e8e45-3c2d-46b6-9ade-d390b56e4a75 nodeName:}" failed. No retries permitted until 2026-10-30 12:56:56.755378282 +0000 UTC m=+34.155431625 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/b86e8e45-3c2d-46b6-9ade-d390b56e4a75-registry-tls") pod "image-registry-69c87564f5-n6wjk" (UID: "b86e8e45-3c2d-46b6-9ade-d390b56e4a75") : secret "image-registry-tls" not found
Oct 30 12:56:54.755952 ip-10-0-142-84 kubenswrapper[2573]: E1030 12:56:54.755826    2573 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Oct 30 12:56:54.755952 ip-10-0-142-84 kubenswrapper[2573]: E1030 12:56:54.755877    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/19b38c46-bfd4-4ed0-a764-f7b672f18cc6-cluster-monitoring-operator-tls podName:19b38c46-bfd4-4ed0-a764-f7b672f18cc6 nodeName:}" failed. No retries permitted until 2026-10-30 12:56:56.755862339 +0000 UTC m=+34.155915680 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/19b38c46-bfd4-4ed0-a764-f7b672f18cc6-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-2sfj9" (UID: "19b38c46-bfd4-4ed0-a764-f7b672f18cc6") : secret "cluster-monitoring-operator-tls" not found
Oct 30 12:56:54.856544 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:54.856504    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e1f629e6-6a96-444f-8329-e71bb98c9029-service-ca-bundle\") pod \"router-default-649bdc978f-lkpqz\" (UID: \"e1f629e6-6a96-444f-8329-e71bb98c9029\") " pod="openshift-ingress/router-default-649bdc978f-lkpqz"
Oct 30 12:56:54.856699 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:54.856583    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/4883c26d-0877-4870-b0ee-406f5bfa8cf4-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-vsqgb\" (UID: \"4883c26d-0877-4870-b0ee-406f5bfa8cf4\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-vsqgb"
Oct 30 12:56:54.856699 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:54.856606    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e1f629e6-6a96-444f-8329-e71bb98c9029-metrics-certs\") pod \"router-default-649bdc978f-lkpqz\" (UID: \"e1f629e6-6a96-444f-8329-e71bb98c9029\") " pod="openshift-ingress/router-default-649bdc978f-lkpqz"
Oct 30 12:56:54.856699 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:54.856632    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1c405568-cb88-4485-ae2c-3614cc31ced6-registry-tls\") pod \"image-registry-77c6ff895d-vw6c6\" (UID: \"1c405568-cb88-4485-ae2c-3614cc31ced6\") " pod="openshift-image-registry/image-registry-77c6ff895d-vw6c6"
Oct 30 12:56:54.856815 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:54.856703    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/23297e74-48f9-47b7-bb5a-80189bdbf2c4-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-7dtcd\" (UID: \"23297e74-48f9-47b7-bb5a-80189bdbf2c4\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-7dtcd"
Oct 30 12:56:54.856848 ip-10-0-142-84 kubenswrapper[2573]: E1030 12:56:54.856829    2573 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Oct 30 12:56:54.856878 ip-10-0-142-84 kubenswrapper[2573]: E1030 12:56:54.856873    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/23297e74-48f9-47b7-bb5a-80189bdbf2c4-samples-operator-tls podName:23297e74-48f9-47b7-bb5a-80189bdbf2c4 nodeName:}" failed. No retries permitted until 2026-10-30 12:56:56.856861974 +0000 UTC m=+34.256915313 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/23297e74-48f9-47b7-bb5a-80189bdbf2c4-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-7dtcd" (UID: "23297e74-48f9-47b7-bb5a-80189bdbf2c4") : secret "samples-operator-tls" not found
Oct 30 12:56:54.857228 ip-10-0-142-84 kubenswrapper[2573]: E1030 12:56:54.857213    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e1f629e6-6a96-444f-8329-e71bb98c9029-service-ca-bundle podName:e1f629e6-6a96-444f-8329-e71bb98c9029 nodeName:}" failed. No retries permitted until 2026-10-30 12:56:56.857202538 +0000 UTC m=+34.257255877 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/e1f629e6-6a96-444f-8329-e71bb98c9029-service-ca-bundle") pod "router-default-649bdc978f-lkpqz" (UID: "e1f629e6-6a96-444f-8329-e71bb98c9029") : configmap references non-existent config key: service-ca.crt
Oct 30 12:56:54.857300 ip-10-0-142-84 kubenswrapper[2573]: E1030 12:56:54.857280    2573 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Oct 30 12:56:54.857355 ip-10-0-142-84 kubenswrapper[2573]: E1030 12:56:54.857305    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4883c26d-0877-4870-b0ee-406f5bfa8cf4-networking-console-plugin-cert podName:4883c26d-0877-4870-b0ee-406f5bfa8cf4 nodeName:}" failed. No retries permitted until 2026-10-30 12:56:56.857297931 +0000 UTC m=+34.257351269 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/4883c26d-0877-4870-b0ee-406f5bfa8cf4-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-vsqgb" (UID: "4883c26d-0877-4870-b0ee-406f5bfa8cf4") : secret "networking-console-plugin-cert" not found
Oct 30 12:56:54.857417 ip-10-0-142-84 kubenswrapper[2573]: E1030 12:56:54.857355    2573 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Oct 30 12:56:54.857417 ip-10-0-142-84 kubenswrapper[2573]: E1030 12:56:54.857388    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e1f629e6-6a96-444f-8329-e71bb98c9029-metrics-certs podName:e1f629e6-6a96-444f-8329-e71bb98c9029 nodeName:}" failed. No retries permitted until 2026-10-30 12:56:56.857376739 +0000 UTC m=+34.257430083 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e1f629e6-6a96-444f-8329-e71bb98c9029-metrics-certs") pod "router-default-649bdc978f-lkpqz" (UID: "e1f629e6-6a96-444f-8329-e71bb98c9029") : secret "router-metrics-certs-default" not found
Oct 30 12:56:54.857511 ip-10-0-142-84 kubenswrapper[2573]: E1030 12:56:54.857448    2573 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Oct 30 12:56:54.857511 ip-10-0-142-84 kubenswrapper[2573]: E1030 12:56:54.857459    2573 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-77c6ff895d-vw6c6: secret "image-registry-tls" not found
Oct 30 12:56:54.857511 ip-10-0-142-84 kubenswrapper[2573]: E1030 12:56:54.857491    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1c405568-cb88-4485-ae2c-3614cc31ced6-registry-tls podName:1c405568-cb88-4485-ae2c-3614cc31ced6 nodeName:}" failed. No retries permitted until 2026-10-30 12:56:56.857481306 +0000 UTC m=+34.257534648 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/1c405568-cb88-4485-ae2c-3614cc31ced6-registry-tls") pod "image-registry-77c6ff895d-vw6c6" (UID: "1c405568-cb88-4485-ae2c-3614cc31ced6") : secret "image-registry-tls" not found
Oct 30 12:56:54.957466 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:54.957436    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/fcf99170-f5cf-4987-97bd-1c30b1b37a35-metrics-tls\") pod \"dns-default-g2ckh\" (UID: \"fcf99170-f5cf-4987-97bd-1c30b1b37a35\") " pod="openshift-dns/dns-default-g2ckh"
Oct 30 12:56:54.957755 ip-10-0-142-84 kubenswrapper[2573]: E1030 12:56:54.957737    2573 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Oct 30 12:56:54.957857 ip-10-0-142-84 kubenswrapper[2573]: E1030 12:56:54.957796    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fcf99170-f5cf-4987-97bd-1c30b1b37a35-metrics-tls podName:fcf99170-f5cf-4987-97bd-1c30b1b37a35 nodeName:}" failed. No retries permitted until 2026-10-30 12:56:56.957780376 +0000 UTC m=+34.357833714 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/fcf99170-f5cf-4987-97bd-1c30b1b37a35-metrics-tls") pod "dns-default-g2ckh" (UID: "fcf99170-f5cf-4987-97bd-1c30b1b37a35") : secret "dns-default-metrics-tls" not found
Oct 30 12:56:56.471858 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:56.471818    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-xtfpg" event={"ID":"f07abb3f-8900-4fa5-8d35-c7ab1d35582d","Type":"ContainerStarted","Data":"8354291f2eb8a1bcdb4f797027d7ee07214438bf0c9da3da24e776b80fee2a04"}
Oct 30 12:56:56.514347 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:56.514058    2573 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-xtfpg" podStartSLOduration=368.521993057 podStartE2EDuration="6m10.514039097s" totalImagesPullingTime="1.99204604s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 12:50:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 12:56:56.51235616 +0000 UTC m=+33.912409546" watchObservedRunningTime="2026-10-30 12:56:56.514039097 +0000 UTC m=+33.914092438"
Oct 30 12:56:56.775490 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:56.775447    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/19b38c46-bfd4-4ed0-a764-f7b672f18cc6-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-2sfj9\" (UID: \"19b38c46-bfd4-4ed0-a764-f7b672f18cc6\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-2sfj9"
Oct 30 12:56:56.775647 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:56.775556    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b86e8e45-3c2d-46b6-9ade-d390b56e4a75-registry-tls\") pod \"image-registry-69c87564f5-n6wjk\" (UID: \"b86e8e45-3c2d-46b6-9ade-d390b56e4a75\") " pod="openshift-image-registry/image-registry-69c87564f5-n6wjk"
Oct 30 12:56:56.775855 ip-10-0-142-84 kubenswrapper[2573]: E1030 12:56:56.775836    2573 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Oct 30 12:56:56.775940 ip-10-0-142-84 kubenswrapper[2573]: E1030 12:56:56.775911    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/19b38c46-bfd4-4ed0-a764-f7b672f18cc6-cluster-monitoring-operator-tls podName:19b38c46-bfd4-4ed0-a764-f7b672f18cc6 nodeName:}" failed. No retries permitted until 2026-10-30 12:57:00.775891149 +0000 UTC m=+38.175944507 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/19b38c46-bfd4-4ed0-a764-f7b672f18cc6-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-2sfj9" (UID: "19b38c46-bfd4-4ed0-a764-f7b672f18cc6") : secret "cluster-monitoring-operator-tls" not found
Oct 30 12:56:56.776644 ip-10-0-142-84 kubenswrapper[2573]: E1030 12:56:56.776349    2573 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Oct 30 12:56:56.776644 ip-10-0-142-84 kubenswrapper[2573]: E1030 12:56:56.776369    2573 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-69c87564f5-n6wjk: secret "image-registry-tls" not found
Oct 30 12:56:56.776644 ip-10-0-142-84 kubenswrapper[2573]: E1030 12:56:56.776434    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b86e8e45-3c2d-46b6-9ade-d390b56e4a75-registry-tls podName:b86e8e45-3c2d-46b6-9ade-d390b56e4a75 nodeName:}" failed. No retries permitted until 2026-10-30 12:57:00.776417832 +0000 UTC m=+38.176471185 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/b86e8e45-3c2d-46b6-9ade-d390b56e4a75-registry-tls") pod "image-registry-69c87564f5-n6wjk" (UID: "b86e8e45-3c2d-46b6-9ade-d390b56e4a75") : secret "image-registry-tls" not found
Oct 30 12:56:56.876591 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:56.876477    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/4883c26d-0877-4870-b0ee-406f5bfa8cf4-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-vsqgb\" (UID: \"4883c26d-0877-4870-b0ee-406f5bfa8cf4\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-vsqgb"
Oct 30 12:56:56.876591 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:56.876515    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e1f629e6-6a96-444f-8329-e71bb98c9029-metrics-certs\") pod \"router-default-649bdc978f-lkpqz\" (UID: \"e1f629e6-6a96-444f-8329-e71bb98c9029\") " pod="openshift-ingress/router-default-649bdc978f-lkpqz"
Oct 30 12:56:56.876591 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:56.876534    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1c405568-cb88-4485-ae2c-3614cc31ced6-registry-tls\") pod \"image-registry-77c6ff895d-vw6c6\" (UID: \"1c405568-cb88-4485-ae2c-3614cc31ced6\") " pod="openshift-image-registry/image-registry-77c6ff895d-vw6c6"
Oct 30 12:56:56.876591 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:56.876582    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/23297e74-48f9-47b7-bb5a-80189bdbf2c4-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-7dtcd\" (UID: \"23297e74-48f9-47b7-bb5a-80189bdbf2c4\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-7dtcd"
Oct 30 12:56:56.876907 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:56.876636    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e1f629e6-6a96-444f-8329-e71bb98c9029-service-ca-bundle\") pod \"router-default-649bdc978f-lkpqz\" (UID: \"e1f629e6-6a96-444f-8329-e71bb98c9029\") " pod="openshift-ingress/router-default-649bdc978f-lkpqz"
Oct 30 12:56:56.876907 ip-10-0-142-84 kubenswrapper[2573]: E1030 12:56:56.876767    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e1f629e6-6a96-444f-8329-e71bb98c9029-service-ca-bundle podName:e1f629e6-6a96-444f-8329-e71bb98c9029 nodeName:}" failed. No retries permitted until 2026-10-30 12:57:00.876753983 +0000 UTC m=+38.276807334 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/e1f629e6-6a96-444f-8329-e71bb98c9029-service-ca-bundle") pod "router-default-649bdc978f-lkpqz" (UID: "e1f629e6-6a96-444f-8329-e71bb98c9029") : configmap references non-existent config key: service-ca.crt
Oct 30 12:56:56.877388 ip-10-0-142-84 kubenswrapper[2573]: E1030 12:56:56.877062    2573 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Oct 30 12:56:56.877388 ip-10-0-142-84 kubenswrapper[2573]: E1030 12:56:56.877102    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4883c26d-0877-4870-b0ee-406f5bfa8cf4-networking-console-plugin-cert podName:4883c26d-0877-4870-b0ee-406f5bfa8cf4 nodeName:}" failed. No retries permitted until 2026-10-30 12:57:00.877091513 +0000 UTC m=+38.277144851 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/4883c26d-0877-4870-b0ee-406f5bfa8cf4-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-vsqgb" (UID: "4883c26d-0877-4870-b0ee-406f5bfa8cf4") : secret "networking-console-plugin-cert" not found
Oct 30 12:56:56.877388 ip-10-0-142-84 kubenswrapper[2573]: E1030 12:56:56.877142    2573 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Oct 30 12:56:56.877388 ip-10-0-142-84 kubenswrapper[2573]: E1030 12:56:56.877160    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e1f629e6-6a96-444f-8329-e71bb98c9029-metrics-certs podName:e1f629e6-6a96-444f-8329-e71bb98c9029 nodeName:}" failed. No retries permitted until 2026-10-30 12:57:00.877154393 +0000 UTC m=+38.277207744 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e1f629e6-6a96-444f-8329-e71bb98c9029-metrics-certs") pod "router-default-649bdc978f-lkpqz" (UID: "e1f629e6-6a96-444f-8329-e71bb98c9029") : secret "router-metrics-certs-default" not found
Oct 30 12:56:56.877388 ip-10-0-142-84 kubenswrapper[2573]: E1030 12:56:56.877205    2573 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Oct 30 12:56:56.877388 ip-10-0-142-84 kubenswrapper[2573]: E1030 12:56:56.877214    2573 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-77c6ff895d-vw6c6: secret "image-registry-tls" not found
Oct 30 12:56:56.877388 ip-10-0-142-84 kubenswrapper[2573]: E1030 12:56:56.877241    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1c405568-cb88-4485-ae2c-3614cc31ced6-registry-tls podName:1c405568-cb88-4485-ae2c-3614cc31ced6 nodeName:}" failed. No retries permitted until 2026-10-30 12:57:00.877234333 +0000 UTC m=+38.277287687 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/1c405568-cb88-4485-ae2c-3614cc31ced6-registry-tls") pod "image-registry-77c6ff895d-vw6c6" (UID: "1c405568-cb88-4485-ae2c-3614cc31ced6") : secret "image-registry-tls" not found
Oct 30 12:56:56.877388 ip-10-0-142-84 kubenswrapper[2573]: E1030 12:56:56.877291    2573 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Oct 30 12:56:56.877388 ip-10-0-142-84 kubenswrapper[2573]: E1030 12:56:56.877311    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/23297e74-48f9-47b7-bb5a-80189bdbf2c4-samples-operator-tls podName:23297e74-48f9-47b7-bb5a-80189bdbf2c4 nodeName:}" failed. No retries permitted until 2026-10-30 12:57:00.877304461 +0000 UTC m=+38.277357812 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/23297e74-48f9-47b7-bb5a-80189bdbf2c4-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-7dtcd" (UID: "23297e74-48f9-47b7-bb5a-80189bdbf2c4") : secret "samples-operator-tls" not found
Oct 30 12:56:56.979087 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:56.977494    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b11d3d97-ad88-4aa3-aaee-e83e3cf12f6d-metrics-certs\") pod \"network-metrics-daemon-p7tv6\" (UID: \"b11d3d97-ad88-4aa3-aaee-e83e3cf12f6d\") " pod="openshift-multus/network-metrics-daemon-p7tv6"
Oct 30 12:56:56.979087 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:56.977568    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9vr7r\" (UniqueName: \"kubernetes.io/projected/8d767e44-14df-4ea6-b391-bdf265773535-kube-api-access-9vr7r\") pod \"network-check-target-l5mdf\" (UID: \"8d767e44-14df-4ea6-b391-bdf265773535\") " pod="openshift-network-diagnostics/network-check-target-l5mdf"
Oct 30 12:56:56.979087 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:56.977609    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f660a256-a12d-4a4a-823d-4b0ff5dcb3da-original-pull-secret\") pod \"global-pull-secret-syncer-rpdcd\" (UID: \"f660a256-a12d-4a4a-823d-4b0ff5dcb3da\") " pod="kube-system/global-pull-secret-syncer-rpdcd"
Oct 30 12:56:56.979087 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:56.977703    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/fcf99170-f5cf-4987-97bd-1c30b1b37a35-metrics-tls\") pod \"dns-default-g2ckh\" (UID: \"fcf99170-f5cf-4987-97bd-1c30b1b37a35\") " pod="openshift-dns/dns-default-g2ckh"
Oct 30 12:56:56.979087 ip-10-0-142-84 kubenswrapper[2573]: E1030 12:56:56.977854    2573 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Oct 30 12:56:56.979087 ip-10-0-142-84 kubenswrapper[2573]: E1030 12:56:56.977911    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fcf99170-f5cf-4987-97bd-1c30b1b37a35-metrics-tls podName:fcf99170-f5cf-4987-97bd-1c30b1b37a35 nodeName:}" failed. No retries permitted until 2026-10-30 12:57:00.977894586 +0000 UTC m=+38.377947927 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/fcf99170-f5cf-4987-97bd-1c30b1b37a35-metrics-tls") pod "dns-default-g2ckh" (UID: "fcf99170-f5cf-4987-97bd-1c30b1b37a35") : secret "dns-default-metrics-tls" not found
Oct 30 12:56:56.979087 ip-10-0-142-84 kubenswrapper[2573]: E1030 12:56:56.978248    2573 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Oct 30 12:56:56.979087 ip-10-0-142-84 kubenswrapper[2573]: E1030 12:56:56.978315    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b11d3d97-ad88-4aa3-aaee-e83e3cf12f6d-metrics-certs podName:b11d3d97-ad88-4aa3-aaee-e83e3cf12f6d nodeName:}" failed. No retries permitted until 2026-10-30 12:57:28.978299771 +0000 UTC m=+66.378353113 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b11d3d97-ad88-4aa3-aaee-e83e3cf12f6d-metrics-certs") pod "network-metrics-daemon-p7tv6" (UID: "b11d3d97-ad88-4aa3-aaee-e83e3cf12f6d") : secret "metrics-daemon-secret" not found
Oct 30 12:56:56.983486 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:56.983416    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9vr7r\" (UniqueName: \"kubernetes.io/projected/8d767e44-14df-4ea6-b391-bdf265773535-kube-api-access-9vr7r\") pod \"network-check-target-l5mdf\" (UID: \"8d767e44-14df-4ea6-b391-bdf265773535\") " pod="openshift-network-diagnostics/network-check-target-l5mdf"
Oct 30 12:56:56.996234 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:56.996206    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f660a256-a12d-4a4a-823d-4b0ff5dcb3da-original-pull-secret\") pod \"global-pull-secret-syncer-rpdcd\" (UID: \"f660a256-a12d-4a4a-823d-4b0ff5dcb3da\") " pod="kube-system/global-pull-secret-syncer-rpdcd"
Oct 30 12:56:57.092731 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:57.092497    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l5mdf"
Oct 30 12:56:57.105167 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:57.105094    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rpdcd"
Oct 30 12:56:57.439965 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:57.439933    2573 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-l5mdf"]
Oct 30 12:56:57.447771 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:56:57.447732    2573 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8d767e44_14df_4ea6_b391_bdf265773535.slice/crio-3389cdf237dd6b19ba280bf5738bd8a7a90c3f3d03e56b7ea402b586ead69d51 WatchSource:0}: Error finding container 3389cdf237dd6b19ba280bf5738bd8a7a90c3f3d03e56b7ea402b586ead69d51: Status 404 returned error can't find the container with id 3389cdf237dd6b19ba280bf5738bd8a7a90c3f3d03e56b7ea402b586ead69d51
Oct 30 12:56:57.477192 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:57.477156    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-l5mdf" event={"ID":"8d767e44-14df-4ea6-b391-bdf265773535","Type":"ContainerStarted","Data":"3389cdf237dd6b19ba280bf5738bd8a7a90c3f3d03e56b7ea402b586ead69d51"}
Oct 30 12:56:57.485893 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:57.485866    2573 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-rpdcd"]
Oct 30 12:56:58.481650 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:58.481546    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-rpdcd" event={"ID":"f660a256-a12d-4a4a-823d-4b0ff5dcb3da","Type":"ContainerStarted","Data":"78bd4381c30eb75c22521d05e237a70dfabe47221dacc0c74ebcbd5e36ff8fbf"}
Oct 30 12:56:59.493762 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:59.493393    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-6f78d59f99-vdwgx" event={"ID":"7ed65395-2cbd-46bf-81dc-fe307f3a9345","Type":"ContainerStarted","Data":"aeaba172322835d333744382dc3dde2c515c22be8929880366d08ecf14907e98"}
Oct 30 12:56:59.497863 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:59.496291    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-8fqfr_d8930ee7-0ae0-48b5-8ce3-cb8820355cef/console-operator/0.log"
Oct 30 12:56:59.497863 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:59.496329    2573 generic.go:356] "Generic (PLEG): container finished" podID="d8930ee7-0ae0-48b5-8ce3-cb8820355cef" containerID="cbab2e610af2ffb6fcb1cf4d641cde326928c18e1cc80dcff6f1d69d7bb4e66a" exitCode=255
Oct 30 12:56:59.497863 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:59.496394    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7cd56b8dbc-8fqfr" event={"ID":"d8930ee7-0ae0-48b5-8ce3-cb8820355cef","Type":"ContainerDied","Data":"cbab2e610af2ffb6fcb1cf4d641cde326928c18e1cc80dcff6f1d69d7bb4e66a"}
Oct 30 12:56:59.497863 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:59.496688    2573 scope.go:122] "RemoveContainer" containerID="cbab2e610af2ffb6fcb1cf4d641cde326928c18e1cc80dcff6f1d69d7bb4e66a"
Oct 30 12:56:59.503285 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:59.500499    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-plmkt" event={"ID":"85f4c399-0089-4399-86e2-069834f89286","Type":"ContainerStarted","Data":"095c72e4ebb12fd5c0f400652dd933d4d6ee0b1e134430ae1bec9fde18a6aa98"}
Oct 30 12:56:59.523022 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:59.522502    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-h42v2" event={"ID":"27c55f84-4e9a-44de-94ea-02258d6bc747","Type":"ContainerStarted","Data":"be9e4b58df52832bf9f3009bd3440bc1f53c0e0245716f42bfe0301d5b3e2141"}
Oct 30 12:56:59.527292 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:59.526878    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-bdf5s" event={"ID":"98e7e1f3-2ca6-45de-a5d8-d27c22e257b1","Type":"ContainerStarted","Data":"2a196efcacef1b4f880c689b8297b0a44ab70e6ad4828703d386d76008c61406"}
Oct 30 12:56:59.533677 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:59.532775    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-7c4986d449-7cfvt" event={"ID":"bd9e55af-64ad-4ffc-92c3-53b7bf654322","Type":"ContainerStarted","Data":"e77daa452696e3d2aec1881194c78a39904af7eed8db1d49200f93af317a9960"}
Oct 30 12:56:59.555235 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:59.555186    2573 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-6f78d59f99-vdwgx" podStartSLOduration=347.184744497 podStartE2EDuration="5m52.555168541s" totalImagesPullingTime="5.370424044s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 12:51:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 12:56:59.549010466 +0000 UTC m=+36.949063845" watchObservedRunningTime="2026-10-30 12:56:59.555168541 +0000 UTC m=+36.955221903"
Oct 30 12:56:59.590280 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:59.590219    2573 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-plmkt" podStartSLOduration=366.478505934 podStartE2EDuration="6m11.590207561s" totalImagesPullingTime="5.111701627s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 12:50:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 12:56:59.589377498 +0000 UTC m=+36.989430859" watchObservedRunningTime="2026-10-30 12:56:59.590207561 +0000 UTC m=+36.990260916"
Oct 30 12:56:59.728222 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:59.724954    2573 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-insights/insights-operator-7c4986d449-7cfvt" podStartSLOduration=368.533389315 podStartE2EDuration="6m13.724936917s" totalImagesPullingTime="5.191547602s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 12:50:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 12:56:59.680515061 +0000 UTC m=+37.080568425" watchObservedRunningTime="2026-10-30 12:56:59.724936917 +0000 UTC m=+37.124990277"
Oct 30 12:56:59.766547 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:59.766495    2573 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-bdf5s" podStartSLOduration=2.76928359 podStartE2EDuration="7.766480927s" totalImagesPullingTime="4.997197337s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 12:56:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 12:56:59.765678649 +0000 UTC m=+37.165732008" watchObservedRunningTime="2026-10-30 12:56:59.766480927 +0000 UTC m=+37.166534286"
Oct 30 12:56:59.768346 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:59.768307    2573 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-h42v2" podStartSLOduration=365.643357863 podStartE2EDuration="6m10.768297857s" totalImagesPullingTime="5.124939994s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 12:50:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 12:56:59.725568007 +0000 UTC m=+37.125621372" watchObservedRunningTime="2026-10-30 12:56:59.768297857 +0000 UTC m=+37.168351216"
Oct 30 12:56:59.936373 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:59.935680    2573 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-knqz6"]
Oct 30 12:56:59.950971 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:59.950946    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-knqz6"
Oct 30 12:56:59.973843 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:59.973767    2573 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-knqz6"]
Oct 30 12:56:59.984793 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:59.984773    2573 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Oct 30 12:56:59.985325 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:59.985288    2573 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-kmvh7\""
Oct 30 12:56:59.985701 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:56:59.985656    2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Oct 30 12:57:00.006630 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:57:00.006285    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/55fa4842-3eb4-4e52-a4fc-18876ca409d5-crio-socket\") pod \"insights-runtime-extractor-knqz6\" (UID: \"55fa4842-3eb4-4e52-a4fc-18876ca409d5\") " pod="openshift-insights/insights-runtime-extractor-knqz6"
Oct 30 12:57:00.006630 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:57:00.006350    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zg85t\" (UniqueName: \"kubernetes.io/projected/55fa4842-3eb4-4e52-a4fc-18876ca409d5-kube-api-access-zg85t\") pod \"insights-runtime-extractor-knqz6\" (UID: \"55fa4842-3eb4-4e52-a4fc-18876ca409d5\") " pod="openshift-insights/insights-runtime-extractor-knqz6"
Oct 30 12:57:00.006630 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:57:00.006387    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/55fa4842-3eb4-4e52-a4fc-18876ca409d5-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-knqz6\" (UID: \"55fa4842-3eb4-4e52-a4fc-18876ca409d5\") " pod="openshift-insights/insights-runtime-extractor-knqz6"
Oct 30 12:57:00.006630 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:57:00.006540    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/55fa4842-3eb4-4e52-a4fc-18876ca409d5-data-volume\") pod \"insights-runtime-extractor-knqz6\" (UID: \"55fa4842-3eb4-4e52-a4fc-18876ca409d5\") " pod="openshift-insights/insights-runtime-extractor-knqz6"
Oct 30 12:57:00.006630 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:57:00.006570    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/55fa4842-3eb4-4e52-a4fc-18876ca409d5-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-knqz6\" (UID: \"55fa4842-3eb4-4e52-a4fc-18876ca409d5\") " pod="openshift-insights/insights-runtime-extractor-knqz6"
Oct 30 12:57:00.108877 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:57:00.107530    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/55fa4842-3eb4-4e52-a4fc-18876ca409d5-crio-socket\") pod \"insights-runtime-extractor-knqz6\" (UID: \"55fa4842-3eb4-4e52-a4fc-18876ca409d5\") " pod="openshift-insights/insights-runtime-extractor-knqz6"
Oct 30 12:57:00.108877 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:57:00.107613    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zg85t\" (UniqueName: \"kubernetes.io/projected/55fa4842-3eb4-4e52-a4fc-18876ca409d5-kube-api-access-zg85t\") pod \"insights-runtime-extractor-knqz6\" (UID: \"55fa4842-3eb4-4e52-a4fc-18876ca409d5\") " pod="openshift-insights/insights-runtime-extractor-knqz6"
Oct 30 12:57:00.108877 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:57:00.107646    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/55fa4842-3eb4-4e52-a4fc-18876ca409d5-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-knqz6\" (UID: \"55fa4842-3eb4-4e52-a4fc-18876ca409d5\") " pod="openshift-insights/insights-runtime-extractor-knqz6"
Oct 30 12:57:00.108877 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:57:00.107726    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/55fa4842-3eb4-4e52-a4fc-18876ca409d5-data-volume\") pod \"insights-runtime-extractor-knqz6\" (UID: \"55fa4842-3eb4-4e52-a4fc-18876ca409d5\") " pod="openshift-insights/insights-runtime-extractor-knqz6"
Oct 30 12:57:00.108877 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:57:00.107749    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/55fa4842-3eb4-4e52-a4fc-18876ca409d5-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-knqz6\" (UID: \"55fa4842-3eb4-4e52-a4fc-18876ca409d5\") " pod="openshift-insights/insights-runtime-extractor-knqz6"
Oct 30 12:57:00.108877 ip-10-0-142-84 kubenswrapper[2573]: E1030 12:57:00.107963    2573 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Oct 30 12:57:00.108877 ip-10-0-142-84 kubenswrapper[2573]: E1030 12:57:00.108029    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/55fa4842-3eb4-4e52-a4fc-18876ca409d5-insights-runtime-extractor-tls podName:55fa4842-3eb4-4e52-a4fc-18876ca409d5 nodeName:}" failed. No retries permitted until 2026-10-30 12:57:00.608010983 +0000 UTC m=+38.008064341 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/55fa4842-3eb4-4e52-a4fc-18876ca409d5-insights-runtime-extractor-tls") pod "insights-runtime-extractor-knqz6" (UID: "55fa4842-3eb4-4e52-a4fc-18876ca409d5") : secret "insights-runtime-extractor-tls" not found
Oct 30 12:57:00.108877 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:57:00.108361    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/55fa4842-3eb4-4e52-a4fc-18876ca409d5-crio-socket\") pod \"insights-runtime-extractor-knqz6\" (UID: \"55fa4842-3eb4-4e52-a4fc-18876ca409d5\") " pod="openshift-insights/insights-runtime-extractor-knqz6"
Oct 30 12:57:00.108877 ip-10-0-142-84 kubenswrapper[2573]: E1030 12:57:00.108629    2573 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Oct 30 12:57:00.108877 ip-10-0-142-84 kubenswrapper[2573]: E1030 12:57:00.108676    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/55fa4842-3eb4-4e52-a4fc-18876ca409d5-kube-rbac-proxy-cm podName:55fa4842-3eb4-4e52-a4fc-18876ca409d5 nodeName:}" failed. No retries permitted until 2026-10-30 12:57:00.608661206 +0000 UTC m=+38.008714558 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/55fa4842-3eb4-4e52-a4fc-18876ca409d5-kube-rbac-proxy-cm") pod "insights-runtime-extractor-knqz6" (UID: "55fa4842-3eb4-4e52-a4fc-18876ca409d5") : configmap "kube-rbac-proxy" not found
Oct 30 12:57:00.109462 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:57:00.108995    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/55fa4842-3eb4-4e52-a4fc-18876ca409d5-data-volume\") pod \"insights-runtime-extractor-knqz6\" (UID: \"55fa4842-3eb4-4e52-a4fc-18876ca409d5\") " pod="openshift-insights/insights-runtime-extractor-knqz6"
Oct 30 12:57:00.145834 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:57:00.145383    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zg85t\" (UniqueName: \"kubernetes.io/projected/55fa4842-3eb4-4e52-a4fc-18876ca409d5-kube-api-access-zg85t\") pod \"insights-runtime-extractor-knqz6\" (UID: \"55fa4842-3eb4-4e52-a4fc-18876ca409d5\") " pod="openshift-insights/insights-runtime-extractor-knqz6"
Oct 30 12:57:00.523289 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:57:00.522432    2573 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-66df5d496b-cfhqz"]
Oct 30 12:57:00.539070 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:57:00.539046    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-8fqfr_d8930ee7-0ae0-48b5-8ce3-cb8820355cef/console-operator/1.log"
Oct 30 12:57:00.539623 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:57:00.539601    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-8fqfr_d8930ee7-0ae0-48b5-8ce3-cb8820355cef/console-operator/0.log"
Oct 30 12:57:00.539702 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:57:00.539651    2573 generic.go:356] "Generic (PLEG): container finished" podID="d8930ee7-0ae0-48b5-8ce3-cb8820355cef" containerID="0d08c313aba902fdd2b8020553685be737434486e076384118441c7db6d44b42" exitCode=255
Oct 30 12:57:00.539990 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:57:00.539972    2573 scope.go:122] "RemoveContainer" containerID="0d08c313aba902fdd2b8020553685be737434486e076384118441c7db6d44b42"
Oct 30 12:57:00.540204 ip-10-0-142-84 kubenswrapper[2573]: E1030 12:57:00.540180    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7cd56b8dbc-8fqfr_openshift-console-operator(d8930ee7-0ae0-48b5-8ce3-cb8820355cef)\"" pod="openshift-console-operator/console-operator-7cd56b8dbc-8fqfr" podUID="d8930ee7-0ae0-48b5-8ce3-cb8820355cef"
Oct 30 12:57:00.540753 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:57:00.540561    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7cd56b8dbc-8fqfr" event={"ID":"d8930ee7-0ae0-48b5-8ce3-cb8820355cef","Type":"ContainerDied","Data":"0d08c313aba902fdd2b8020553685be737434486e076384118441c7db6d44b42"}
Oct 30 12:57:00.540753 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:57:00.540625    2573 scope.go:122] "RemoveContainer" containerID="cbab2e610af2ffb6fcb1cf4d641cde326928c18e1cc80dcff6f1d69d7bb4e66a"
Oct 30 12:57:00.540958 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:57:00.540940    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-cfhqz"
Oct 30 12:57:00.546463 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:57:00.546442    2573 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-66df5d496b-cfhqz"]
Oct 30 12:57:00.612392 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:57:00.612363    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qszn6\" (UniqueName: \"kubernetes.io/projected/e30bf7f2-4c3a-4801-8ef1-f0611812a9d4-kube-api-access-qszn6\") pod \"migrator-66df5d496b-cfhqz\" (UID: \"e30bf7f2-4c3a-4801-8ef1-f0611812a9d4\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-cfhqz"
Oct 30 12:57:00.612843 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:57:00.612816    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/55fa4842-3eb4-4e52-a4fc-18876ca409d5-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-knqz6\" (UID: \"55fa4842-3eb4-4e52-a4fc-18876ca409d5\") " pod="openshift-insights/insights-runtime-extractor-knqz6"
Oct 30 12:57:00.612966 ip-10-0-142-84 kubenswrapper[2573]: E1030 12:57:00.612947    2573 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Oct 30 12:57:00.613038 ip-10-0-142-84 kubenswrapper[2573]: E1030 12:57:00.613006    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/55fa4842-3eb4-4e52-a4fc-18876ca409d5-kube-rbac-proxy-cm podName:55fa4842-3eb4-4e52-a4fc-18876ca409d5 nodeName:}" failed. No retries permitted until 2026-10-30 12:57:01.612988079 +0000 UTC m=+39.013041432 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/55fa4842-3eb4-4e52-a4fc-18876ca409d5-kube-rbac-proxy-cm") pod "insights-runtime-extractor-knqz6" (UID: "55fa4842-3eb4-4e52-a4fc-18876ca409d5") : configmap "kube-rbac-proxy" not found
Oct 30 12:57:00.613108 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:57:00.613049    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/55fa4842-3eb4-4e52-a4fc-18876ca409d5-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-knqz6\" (UID: \"55fa4842-3eb4-4e52-a4fc-18876ca409d5\") " pod="openshift-insights/insights-runtime-extractor-knqz6"
Oct 30 12:57:00.613168 ip-10-0-142-84 kubenswrapper[2573]: E1030 12:57:00.613137    2573 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Oct 30 12:57:00.613222 ip-10-0-142-84 kubenswrapper[2573]: E1030 12:57:00.613193    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/55fa4842-3eb4-4e52-a4fc-18876ca409d5-insights-runtime-extractor-tls podName:55fa4842-3eb4-4e52-a4fc-18876ca409d5 nodeName:}" failed. No retries permitted until 2026-10-30 12:57:01.613173339 +0000 UTC m=+39.013226720 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/55fa4842-3eb4-4e52-a4fc-18876ca409d5-insights-runtime-extractor-tls") pod "insights-runtime-extractor-knqz6" (UID: "55fa4842-3eb4-4e52-a4fc-18876ca409d5") : secret "insights-runtime-extractor-tls" not found
Oct 30 12:57:00.651850 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:57:00.651825    2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Oct 30 12:57:00.651949 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:57:00.651862    2573 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-wf8rx\""
Oct 30 12:57:00.652314 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:57:00.652293    2573 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Oct 30 12:57:00.714368 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:57:00.714340    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qszn6\" (UniqueName: \"kubernetes.io/projected/e30bf7f2-4c3a-4801-8ef1-f0611812a9d4-kube-api-access-qszn6\") pod \"migrator-66df5d496b-cfhqz\" (UID: \"e30bf7f2-4c3a-4801-8ef1-f0611812a9d4\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-cfhqz"
Oct 30 12:57:00.739111 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:57:00.739084    2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qszn6\" (UniqueName: \"kubernetes.io/projected/e30bf7f2-4c3a-4801-8ef1-f0611812a9d4-kube-api-access-qszn6\") pod \"migrator-66df5d496b-cfhqz\" (UID: \"e30bf7f2-4c3a-4801-8ef1-f0611812a9d4\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-cfhqz"
Oct 30 12:57:00.815356 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:57:00.815288    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/19b38c46-bfd4-4ed0-a764-f7b672f18cc6-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-2sfj9\" (UID: \"19b38c46-bfd4-4ed0-a764-f7b672f18cc6\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-2sfj9"
Oct 30 12:57:00.815508 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:57:00.815389    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b86e8e45-3c2d-46b6-9ade-d390b56e4a75-registry-tls\") pod \"image-registry-69c87564f5-n6wjk\" (UID: \"b86e8e45-3c2d-46b6-9ade-d390b56e4a75\") " pod="openshift-image-registry/image-registry-69c87564f5-n6wjk"
Oct 30 12:57:00.815508 ip-10-0-142-84 kubenswrapper[2573]: E1030 12:57:00.815444    2573 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Oct 30 12:57:00.815614 ip-10-0-142-84 kubenswrapper[2573]: E1030 12:57:00.815531    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/19b38c46-bfd4-4ed0-a764-f7b672f18cc6-cluster-monitoring-operator-tls podName:19b38c46-bfd4-4ed0-a764-f7b672f18cc6 nodeName:}" failed. No retries permitted until 2026-10-30 12:57:08.815509653 +0000 UTC m=+46.215563000 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/19b38c46-bfd4-4ed0-a764-f7b672f18cc6-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-2sfj9" (UID: "19b38c46-bfd4-4ed0-a764-f7b672f18cc6") : secret "cluster-monitoring-operator-tls" not found
Oct 30 12:57:00.815614 ip-10-0-142-84 kubenswrapper[2573]: E1030 12:57:00.815585    2573 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Oct 30 12:57:00.815614 ip-10-0-142-84 kubenswrapper[2573]: E1030 12:57:00.815600    2573 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-69c87564f5-n6wjk: secret "image-registry-tls" not found
Oct 30 12:57:00.815747 ip-10-0-142-84 kubenswrapper[2573]: E1030 12:57:00.815648    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b86e8e45-3c2d-46b6-9ade-d390b56e4a75-registry-tls podName:b86e8e45-3c2d-46b6-9ade-d390b56e4a75 nodeName:}" failed. No retries permitted until 2026-10-30 12:57:08.815633051 +0000 UTC m=+46.215686408 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/b86e8e45-3c2d-46b6-9ade-d390b56e4a75-registry-tls") pod "image-registry-69c87564f5-n6wjk" (UID: "b86e8e45-3c2d-46b6-9ade-d390b56e4a75") : secret "image-registry-tls" not found
Oct 30 12:57:00.884290 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:57:00.884244    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-cfhqz"
Oct 30 12:57:00.916062 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:57:00.916034    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/23297e74-48f9-47b7-bb5a-80189bdbf2c4-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-7dtcd\" (UID: \"23297e74-48f9-47b7-bb5a-80189bdbf2c4\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-7dtcd"
Oct 30 12:57:00.916177 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:57:00.916113    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e1f629e6-6a96-444f-8329-e71bb98c9029-service-ca-bundle\") pod \"router-default-649bdc978f-lkpqz\" (UID: \"e1f629e6-6a96-444f-8329-e71bb98c9029\") " pod="openshift-ingress/router-default-649bdc978f-lkpqz"
Oct 30 12:57:00.916177 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:57:00.916155    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/4883c26d-0877-4870-b0ee-406f5bfa8cf4-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-vsqgb\" (UID: \"4883c26d-0877-4870-b0ee-406f5bfa8cf4\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-vsqgb"
Oct 30 12:57:00.916283 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:57:00.916178    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e1f629e6-6a96-444f-8329-e71bb98c9029-metrics-certs\") pod \"router-default-649bdc978f-lkpqz\" (UID: \"e1f629e6-6a96-444f-8329-e71bb98c9029\") " pod="openshift-ingress/router-default-649bdc978f-lkpqz"
Oct 30 12:57:00.916283 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:57:00.916194    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1c405568-cb88-4485-ae2c-3614cc31ced6-registry-tls\") pod \"image-registry-77c6ff895d-vw6c6\" (UID: \"1c405568-cb88-4485-ae2c-3614cc31ced6\") " pod="openshift-image-registry/image-registry-77c6ff895d-vw6c6"
Oct 30 12:57:00.916283 ip-10-0-142-84 kubenswrapper[2573]: E1030 12:57:00.916220    2573 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Oct 30 12:57:00.916398 ip-10-0-142-84 kubenswrapper[2573]: E1030 12:57:00.916307    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/23297e74-48f9-47b7-bb5a-80189bdbf2c4-samples-operator-tls podName:23297e74-48f9-47b7-bb5a-80189bdbf2c4 nodeName:}" failed. No retries permitted until 2026-10-30 12:57:08.916287681 +0000 UTC m=+46.316341021 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/23297e74-48f9-47b7-bb5a-80189bdbf2c4-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-7dtcd" (UID: "23297e74-48f9-47b7-bb5a-80189bdbf2c4") : secret "samples-operator-tls" not found
Oct 30 12:57:00.916398 ip-10-0-142-84 kubenswrapper[2573]: E1030 12:57:00.916337    2573 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Oct 30 12:57:00.916398 ip-10-0-142-84 kubenswrapper[2573]: E1030 12:57:00.916352    2573 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-77c6ff895d-vw6c6: secret "image-registry-tls" not found
Oct 30 12:57:00.916398 ip-10-0-142-84 kubenswrapper[2573]: E1030 12:57:00.916391    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1c405568-cb88-4485-ae2c-3614cc31ced6-registry-tls podName:1c405568-cb88-4485-ae2c-3614cc31ced6 nodeName:}" failed. No retries permitted until 2026-10-30 12:57:08.916378255 +0000 UTC m=+46.316431593 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/1c405568-cb88-4485-ae2c-3614cc31ced6-registry-tls") pod "image-registry-77c6ff895d-vw6c6" (UID: "1c405568-cb88-4485-ae2c-3614cc31ced6") : secret "image-registry-tls" not found
Oct 30 12:57:00.916544 ip-10-0-142-84 kubenswrapper[2573]: E1030 12:57:00.916460    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e1f629e6-6a96-444f-8329-e71bb98c9029-service-ca-bundle podName:e1f629e6-6a96-444f-8329-e71bb98c9029 nodeName:}" failed. No retries permitted until 2026-10-30 12:57:08.916448864 +0000 UTC m=+46.316502207 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/e1f629e6-6a96-444f-8329-e71bb98c9029-service-ca-bundle") pod "router-default-649bdc978f-lkpqz" (UID: "e1f629e6-6a96-444f-8329-e71bb98c9029") : configmap references non-existent config key: service-ca.crt
Oct 30 12:57:00.916544 ip-10-0-142-84 kubenswrapper[2573]: E1030 12:57:00.916521    2573 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Oct 30 12:57:00.916607 ip-10-0-142-84 kubenswrapper[2573]: E1030 12:57:00.916551    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4883c26d-0877-4870-b0ee-406f5bfa8cf4-networking-console-plugin-cert podName:4883c26d-0877-4870-b0ee-406f5bfa8cf4 nodeName:}" failed. No retries permitted until 2026-10-30 12:57:08.916541095 +0000 UTC m=+46.316594433 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/4883c26d-0877-4870-b0ee-406f5bfa8cf4-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-vsqgb" (UID: "4883c26d-0877-4870-b0ee-406f5bfa8cf4") : secret "networking-console-plugin-cert" not found
Oct 30 12:57:00.916647 ip-10-0-142-84 kubenswrapper[2573]: E1030 12:57:00.916603    2573 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Oct 30 12:57:00.916678 ip-10-0-142-84 kubenswrapper[2573]: E1030 12:57:00.916647    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e1f629e6-6a96-444f-8329-e71bb98c9029-metrics-certs podName:e1f629e6-6a96-444f-8329-e71bb98c9029 nodeName:}" failed. No retries permitted until 2026-10-30 12:57:08.916637877 +0000 UTC m=+46.316691219 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e1f629e6-6a96-444f-8329-e71bb98c9029-metrics-certs") pod "router-default-649bdc978f-lkpqz" (UID: "e1f629e6-6a96-444f-8329-e71bb98c9029") : secret "router-metrics-certs-default" not found
Oct 30 12:57:01.016672 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:57:01.016605    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/fcf99170-f5cf-4987-97bd-1c30b1b37a35-metrics-tls\") pod \"dns-default-g2ckh\" (UID: \"fcf99170-f5cf-4987-97bd-1c30b1b37a35\") " pod="openshift-dns/dns-default-g2ckh"
Oct 30 12:57:01.016872 ip-10-0-142-84 kubenswrapper[2573]: E1030 12:57:01.016782    2573 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Oct 30 12:57:01.016872 ip-10-0-142-84 kubenswrapper[2573]: E1030 12:57:01.016859    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fcf99170-f5cf-4987-97bd-1c30b1b37a35-metrics-tls podName:fcf99170-f5cf-4987-97bd-1c30b1b37a35 nodeName:}" failed. No retries permitted until 2026-10-30 12:57:09.016837465 +0000 UTC m=+46.416890823 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/fcf99170-f5cf-4987-97bd-1c30b1b37a35-metrics-tls") pod "dns-default-g2ckh" (UID: "fcf99170-f5cf-4987-97bd-1c30b1b37a35") : secret "dns-default-metrics-tls" not found
Oct 30 12:57:01.207991 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:57:01.207958    2573 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-66df5d496b-cfhqz"]
Oct 30 12:57:01.213219 ip-10-0-142-84 kubenswrapper[2573]: W1030 12:57:01.213192    2573 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode30bf7f2_4c3a_4801_8ef1_f0611812a9d4.slice/crio-30f12e40f554fc638dc898f183ece14727f45446ed0a5f886f254f2034e43302 WatchSource:0}: Error finding container 30f12e40f554fc638dc898f183ece14727f45446ed0a5f886f254f2034e43302: Status 404 returned error can't find the container with id 30f12e40f554fc638dc898f183ece14727f45446ed0a5f886f254f2034e43302
Oct 30 12:57:01.545461 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:57:01.545382    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-cfhqz" event={"ID":"e30bf7f2-4c3a-4801-8ef1-f0611812a9d4","Type":"ContainerStarted","Data":"30f12e40f554fc638dc898f183ece14727f45446ed0a5f886f254f2034e43302"}
Oct 30 12:57:01.547164 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:57:01.547137    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-8fqfr_d8930ee7-0ae0-48b5-8ce3-cb8820355cef/console-operator/1.log"
Oct 30 12:57:01.547558 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:57:01.547533    2573 scope.go:122] "RemoveContainer" containerID="0d08c313aba902fdd2b8020553685be737434486e076384118441c7db6d44b42"
Oct 30 12:57:01.547752 ip-10-0-142-84 kubenswrapper[2573]: E1030 12:57:01.547733    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7cd56b8dbc-8fqfr_openshift-console-operator(d8930ee7-0ae0-48b5-8ce3-cb8820355cef)\"" pod="openshift-console-operator/console-operator-7cd56b8dbc-8fqfr" podUID="d8930ee7-0ae0-48b5-8ce3-cb8820355cef"
Oct 30 12:57:01.548910 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:57:01.548886    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-l5mdf" event={"ID":"8d767e44-14df-4ea6-b391-bdf265773535","Type":"ContainerStarted","Data":"28b7e019a51d880006bf8c25daba3f8eaba35a3d36bfd00ef25b982316b95e16"}
Oct 30 12:57:01.549063 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:57:01.549045    2573 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-l5mdf"
Oct 30 12:57:01.623072 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:57:01.623047    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/55fa4842-3eb4-4e52-a4fc-18876ca409d5-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-knqz6\" (UID: \"55fa4842-3eb4-4e52-a4fc-18876ca409d5\") " pod="openshift-insights/insights-runtime-extractor-knqz6"
Oct 30 12:57:01.623210 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:57:01.623157    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/55fa4842-3eb4-4e52-a4fc-18876ca409d5-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-knqz6\" (UID: \"55fa4842-3eb4-4e52-a4fc-18876ca409d5\") " pod="openshift-insights/insights-runtime-extractor-knqz6"
Oct 30 12:57:01.623567 ip-10-0-142-84 kubenswrapper[2573]: E1030 12:57:01.623549    2573 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Oct 30 12:57:01.623660 ip-10-0-142-84 kubenswrapper[2573]: E1030 12:57:01.623597    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/55fa4842-3eb4-4e52-a4fc-18876ca409d5-insights-runtime-extractor-tls podName:55fa4842-3eb4-4e52-a4fc-18876ca409d5 nodeName:}" failed. No retries permitted until 2026-10-30 12:57:03.623582847 +0000 UTC m=+41.023636184 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/55fa4842-3eb4-4e52-a4fc-18876ca409d5-insights-runtime-extractor-tls") pod "insights-runtime-extractor-knqz6" (UID: "55fa4842-3eb4-4e52-a4fc-18876ca409d5") : secret "insights-runtime-extractor-tls" not found
Oct 30 12:57:01.623726 ip-10-0-142-84 kubenswrapper[2573]: E1030 12:57:01.623555    2573 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Oct 30 12:57:01.623726 ip-10-0-142-84 kubenswrapper[2573]: E1030 12:57:01.623717    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/55fa4842-3eb4-4e52-a4fc-18876ca409d5-kube-rbac-proxy-cm podName:55fa4842-3eb4-4e52-a4fc-18876ca409d5 nodeName:}" failed. No retries permitted until 2026-10-30 12:57:03.623699753 +0000 UTC m=+41.023753111 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/55fa4842-3eb4-4e52-a4fc-18876ca409d5-kube-rbac-proxy-cm") pod "insights-runtime-extractor-knqz6" (UID: "55fa4842-3eb4-4e52-a4fc-18876ca409d5") : configmap "kube-rbac-proxy" not found
Oct 30 12:57:01.634380 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:57:01.634346    2573 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-l5mdf" podStartSLOduration=35.581543851 podStartE2EDuration="38.634335588s" totalImagesPullingTime="3.052791737s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 12:56:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 12:57:01.634072124 +0000 UTC m=+39.034125483" watchObservedRunningTime="2026-10-30 12:57:01.634335588 +0000 UTC m=+39.034388947"
Oct 30 12:57:03.445710 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:57:03.445679    2573 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7cd56b8dbc-8fqfr"
Oct 30 12:57:03.446073 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:57:03.445738    2573 kubelet.go:2727] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-7cd56b8dbc-8fqfr"
Oct 30 12:57:03.446073 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:57:03.446035    2573 scope.go:122] "RemoveContainer" containerID="0d08c313aba902fdd2b8020553685be737434486e076384118441c7db6d44b42"
Oct 30 12:57:03.446223 ip-10-0-142-84 kubenswrapper[2573]: E1030 12:57:03.446204    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7cd56b8dbc-8fqfr_openshift-console-operator(d8930ee7-0ae0-48b5-8ce3-cb8820355cef)\"" pod="openshift-console-operator/console-operator-7cd56b8dbc-8fqfr" podUID="d8930ee7-0ae0-48b5-8ce3-cb8820355cef"
Oct 30 12:57:03.557687 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:57:03.557656    2573 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-cfhqz" event={"ID":"e30bf7f2-4c3a-4801-8ef1-f0611812a9d4","Type":"ContainerStarted","Data":"d3693fc814aef901784e864c097b162bb0ed35b2915d5826fbbce52ca4cdfdf0"}
Oct 30 12:57:03.596134 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:57:03.596085    2573 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-cfhqz" podStartSLOduration=1.973161661 podStartE2EDuration="3.596071136s" totalImagesPullingTime="1.622909475s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 12:57:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 12:57:03.595398948 +0000 UTC m=+40.995452308" watchObservedRunningTime="2026-10-30 12:57:03.596071136 +0000 UTC m=+40.996124495"
Oct 30 12:57:03.641317 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:57:03.641291    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/55fa4842-3eb4-4e52-a4fc-18876ca409d5-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-knqz6\" (UID: \"55fa4842-3eb4-4e52-a4fc-18876ca409d5\") " pod="openshift-insights/insights-runtime-extractor-knqz6"
Oct 30 12:57:03.641449 ip-10-0-142-84 kubenswrapper[2573]: I1030 12:57:03.641383    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/55fa4842-3eb4-4e52-a4fc-18876ca409d5-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-knqz6\" (UID: \"55fa4842-3eb4-4e52-a4fc-18876ca409d5\") " pod="openshift-insights/insights-runtime-extractor-knqz6"
Oct 30 12:57:03.641449 ip-10-0-142-84 kubenswrapper[2573]: E1030 12:57:03.641395    2573 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Oct 30 12:57:03.641545 ip-10-0-142-84 kubenswrapper[2573]: E1030 12:57:03.641504    2573 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Oct 30 12:57:03.641598 ip-10-0-142-84 kubenswrapper[2573]: E1030 12:57:03.641542    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/55fa4842-3eb4-4e52-a4fc-18876ca409d5-kube-rbac-proxy-cm podName:55fa4842-3eb4-4e52-a4fc-18876ca409d5 nodeName:}" failed. No retries permitted until 2026-10-30 12:57:07.641517712 +0000 UTC m=+45.041571066 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/55fa4842-3eb4-4e52-a4fc-18876ca409d5-kube-rbac-proxy-cm") pod "insights-runtime-extractor-knqz6" (UID: "55fa4842-3eb4-4e52-a4fc-18876ca409d5") : configmap "kube-rbac-proxy" not found
Oct 30 12:57:03.641652 ip-10-0-142-84 kubenswrapper[2573]: E1030 12:57:03.641610    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/55fa4842-3eb4-4e52-a4fc-18876ca409d5-insights-runtime-extractor-tls podName:55fa4842-3eb4-4e52-a4fc-18876ca409d5 nodeName:}" failed. No retries permitted until 2026-10-30 12:57:07.641598699 +0000 UTC m=+45.041652051 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/55fa4842-3eb4-4e52-a4fc-18876ca409d5-insights-runtime-extractor-tls") pod "insights-runtime-extractor-knqz6" (UID: "55fa4842-3eb4-4e52-a4fc-18876ca409d5") : secret "insights-runtime-extractor-tls" not found
Oct 30 12:57:04.151477 ip-10-0-142-84 systemd[1]: Stopping Kubernetes Kubelet...
Oct 30 12:57:04.171640 ip-10-0-142-84 systemd[1]: kubelet.service: Deactivated successfully.
Oct 30 12:57:04.171819 ip-10-0-142-84 systemd[1]: Stopped Kubernetes Kubelet.
Oct 30 12:57:04.171870 ip-10-0-142-84 systemd[1]: kubelet.service: Consumed 2.915s CPU time.
Oct 30 12:57:04.182828 ip-10-0-142-84 systemd[1]: Starting Kubernetes Kubelet...
Oct 30 12:57:04.255917 ip-10-0-142-84 kubenswrapper[4943]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 30 12:57:04.255917 ip-10-0-142-84 kubenswrapper[4943]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Oct 30 12:57:04.255917 ip-10-0-142-84 kubenswrapper[4943]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 30 12:57:04.255917 ip-10-0-142-84 kubenswrapper[4943]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 30 12:57:04.258218 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.258141    4943 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 12:57:04.258218 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.258215    4943 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 12:57:04.258218 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.258220    4943 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 12:57:04.258445 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.258225    4943 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 12:57:04.258445 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.258230    4943 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 12:57:04.258445 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.258236    4943 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 12:57:04.258445 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.258242    4943 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 12:57:04.258445 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.258247    4943 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 12:57:04.258445 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.258273    4943 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 12:57:04.258445 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.258279    4943 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 12:57:04.258445 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.258283    4943 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 12:57:04.258445 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.258287    4943 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 12:57:04.258445 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.258291    4943 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 12:57:04.258445 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.258295    4943 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 12:57:04.258445 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.258299    4943 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 12:57:04.258445 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.258303    4943 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 12:57:04.258445 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.258307    4943 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 12:57:04.258445 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.258311    4943 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 12:57:04.258445 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.258315    4943 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 12:57:04.258445 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.258319    4943 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 12:57:04.258445 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.258323    4943 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 12:57:04.258445 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.258327    4943 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 12:57:04.259286 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.258332    4943 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 12:57:04.259286 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.258336    4943 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 12:57:04.259286 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.258340    4943 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 12:57:04.259286 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.258345    4943 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 12:57:04.259286 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.258349    4943 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 12:57:04.259286 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.258353    4943 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 12:57:04.259286 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.258359    4943 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 12:57:04.259286 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.258363    4943 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 12:57:04.259286 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.258368    4943 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 12:57:04.259286 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.258373    4943 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 12:57:04.259286 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.258377    4943 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 12:57:04.259286 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.258382    4943 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 12:57:04.259286 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.258386    4943 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 12:57:04.259286 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.258391    4943 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 12:57:04.259286 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.258396    4943 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 12:57:04.259286 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.258400    4943 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 12:57:04.259286 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.258405    4943 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 12:57:04.259286 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.258411    4943 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 12:57:04.259286 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.258415    4943 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 12:57:04.260148 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.258420    4943 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 12:57:04.260148 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.258424    4943 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 12:57:04.260148 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.258428    4943 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 12:57:04.260148 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.258433    4943 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 12:57:04.260148 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.258437    4943 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 12:57:04.260148 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.258441    4943 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 12:57:04.260148 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.258445    4943 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 12:57:04.260148 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.258449    4943 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 12:57:04.260148 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.258453    4943 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 12:57:04.260148 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.258458    4943 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 12:57:04.260148 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.258462    4943 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 12:57:04.260148 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.258467    4943 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 12:57:04.260148 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.258471    4943 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 12:57:04.260148 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.258476    4943 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 12:57:04.260148 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.258480    4943 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 12:57:04.260148 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.258484    4943 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 12:57:04.260148 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.258488    4943 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 12:57:04.260148 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.258493    4943 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 12:57:04.260148 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.258497    4943 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 12:57:04.260148 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.258501    4943 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 12:57:04.261057 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.258506    4943 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 12:57:04.261057 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.258510    4943 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 12:57:04.261057 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.258514    4943 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 12:57:04.261057 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.258519    4943 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 12:57:04.261057 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.258523    4943 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 12:57:04.261057 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.258527    4943 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 12:57:04.261057 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.258531    4943 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 12:57:04.261057 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.258540    4943 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 12:57:04.261057 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.258545    4943 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 12:57:04.261057 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.258549    4943 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 12:57:04.261057 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.258554    4943 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 12:57:04.261057 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.258559    4943 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 12:57:04.261057 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.258564    4943 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 12:57:04.261057 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.258569    4943 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 12:57:04.261057 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.258573    4943 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 12:57:04.261057 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.258577    4943 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 12:57:04.261057 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.258582    4943 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 12:57:04.261057 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.258587    4943 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 12:57:04.261057 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.258594    4943 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 12:57:04.261931 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.258600    4943 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 12:57:04.261931 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.258605    4943 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 12:57:04.261931 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.258609    4943 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 12:57:04.261931 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.258614    4943 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 12:57:04.261931 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.258618    4943 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 12:57:04.261931 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.258623    4943 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 12:57:04.261931 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.258627    4943 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 12:57:04.261931 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.258632    4943 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 12:57:04.261931 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.258636    4943 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 12:57:04.261931 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.258640    4943 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 12:57:04.261931 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.258645    4943 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 12:57:04.261931 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.258649    4943 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 12:57:04.261931 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.258653    4943 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 12:57:04.261931 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.258658    4943 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 12:57:04.261931 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.258662    4943 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 12:57:04.261931 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.258667    4943 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 12:57:04.261931 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.258671    4943 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 12:57:04.261931 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.258676    4943 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 12:57:04.261931 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.258681    4943 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 12:57:04.261931 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.258685    4943 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 12:57:04.262853 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.258691    4943 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 12:57:04.262853 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.258696    4943 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 12:57:04.262853 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.258700    4943 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 12:57:04.262853 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.258705    4943 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 12:57:04.262853 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.258709    4943 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 12:57:04.262853 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.258714    4943 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 12:57:04.262853 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.259996    4943 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 12:57:04.262853 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.260008    4943 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 12:57:04.262853 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.260014    4943 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 12:57:04.262853 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.260018    4943 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 12:57:04.262853 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.260023    4943 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 12:57:04.262853 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.260028    4943 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 12:57:04.262853 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.260032    4943 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 12:57:04.262853 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.260036    4943 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 12:57:04.262853 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.260041    4943 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 12:57:04.262853 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.260048    4943 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 12:57:04.262853 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.260052    4943 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 12:57:04.262853 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.260056    4943 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 12:57:04.262853 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.260060    4943 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 12:57:04.262853 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.260065    4943 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 12:57:04.263820 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.260069    4943 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 12:57:04.263820 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.260073    4943 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 12:57:04.263820 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.260078    4943 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 12:57:04.263820 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.260082    4943 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 12:57:04.263820 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.260086    4943 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 12:57:04.263820 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.260091    4943 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 12:57:04.263820 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.260095    4943 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 12:57:04.263820 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.260099    4943 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 12:57:04.263820 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.260104    4943 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 12:57:04.263820 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.260108    4943 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 12:57:04.263820 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.260113    4943 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 12:57:04.263820 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.260117    4943 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 12:57:04.263820 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.260122    4943 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 12:57:04.263820 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.260126    4943 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 12:57:04.263820 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.260131    4943 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 12:57:04.263820 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.260135    4943 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 12:57:04.263820 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.260139    4943 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 12:57:04.263820 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.260143    4943 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 12:57:04.263820 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.260148    4943 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 12:57:04.264664 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.260153    4943 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 12:57:04.264664 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.260157    4943 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 12:57:04.264664 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.260161    4943 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 12:57:04.264664 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.260166    4943 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 12:57:04.264664 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.260172    4943 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 12:57:04.264664 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.260177    4943 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 12:57:04.264664 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.260182    4943 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 12:57:04.264664 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.260187    4943 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 12:57:04.264664 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.260192    4943 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 12:57:04.264664 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.260196    4943 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 12:57:04.264664 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.260200    4943 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 12:57:04.264664 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.260207    4943 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 12:57:04.264664 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.260213    4943 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 12:57:04.264664 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.260218    4943 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 12:57:04.264664 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.260222    4943 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 12:57:04.264664 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.260226    4943 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 12:57:04.264664 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.260231    4943 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 12:57:04.264664 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.260236    4943 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 12:57:04.264664 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.260241    4943 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 12:57:04.265322 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.260245    4943 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 12:57:04.265322 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.260249    4943 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 12:57:04.265322 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.260256    4943 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 12:57:04.265322 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.260278    4943 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 12:57:04.265322 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.260282    4943 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 12:57:04.265322 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.260287    4943 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 12:57:04.265322 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.260292    4943 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 12:57:04.265322 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.260296    4943 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 12:57:04.265322 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.260301    4943 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 12:57:04.265322 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.260305    4943 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 12:57:04.265322 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.260309    4943 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 12:57:04.265322 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.260313    4943 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 12:57:04.265322 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.260318    4943 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 12:57:04.265322 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.260324    4943 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 12:57:04.265322 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.260328    4943 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 12:57:04.265322 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.260332    4943 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 12:57:04.265322 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.260336    4943 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 12:57:04.265322 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.260341    4943 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 12:57:04.265322 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.260346    4943 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 12:57:04.265322 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.260350    4943 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 12:57:04.265829 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.260355    4943 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 12:57:04.265829 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.260359    4943 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 12:57:04.265829 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.260364    4943 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 12:57:04.265829 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.260368    4943 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 12:57:04.265829 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.260373    4943 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 12:57:04.265829 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.260377    4943 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 12:57:04.265829 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.260382    4943 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 12:57:04.265829 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.260386    4943 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 12:57:04.265829 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.260391    4943 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 12:57:04.265829 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.260396    4943 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 12:57:04.265829 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.260400    4943 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 12:57:04.265829 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.260404    4943 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 12:57:04.265829 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.260409    4943 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 12:57:04.265829 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.260413    4943 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 12:57:04.265829 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.260417    4943 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 12:57:04.265829 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.260422    4943 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 12:57:04.265829 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.260426    4943 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 12:57:04.265829 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.260430    4943 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 12:57:04.265829 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.260441    4943 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 12:57:04.265829 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.260445    4943 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 12:57:04.266405 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.260449    4943 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 12:57:04.266405 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.260454    4943 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 12:57:04.266405 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.260458    4943 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 12:57:04.266405 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.260462    4943 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 12:57:04.266405 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.260467    4943 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 12:57:04.266405 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.260471    4943 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 12:57:04.266405 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.260476    4943 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 12:57:04.266405 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.260480    4943 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 12:57:04.266405 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.260485    4943 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 12:57:04.266405 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.260491    4943 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 12:57:04.266405 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.260496    4943 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 12:57:04.266405 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.260500    4943 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 12:57:04.266405 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.260504    4943 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 12:57:04.266405 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.260509    4943 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 12:57:04.266405 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.261372    4943 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 12:57:04.266405 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.261382    4943 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 12:57:04.266405 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.261387    4943 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 12:57:04.266405 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.261392    4943 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 12:57:04.266405 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.261397    4943 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 12:57:04.266405 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.261402    4943 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 12:57:04.266909 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.261407    4943 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 12:57:04.266909 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.261411    4943 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 12:57:04.266909 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.261415    4943 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 12:57:04.266909 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.261420    4943 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 12:57:04.266909 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.261424    4943 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 12:57:04.266909 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.261429    4943 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 12:57:04.266909 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.261433    4943 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 12:57:04.266909 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.261438    4943 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 12:57:04.266909 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.261442    4943 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 12:57:04.266909 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.261446    4943 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 12:57:04.266909 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.261456    4943 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 12:57:04.266909 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.261463    4943 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 12:57:04.266909 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.261469    4943 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 12:57:04.266909 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.261473    4943 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 12:57:04.266909 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.261477    4943 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 12:57:04.266909 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.261482    4943 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 12:57:04.266909 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.261486    4943 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 12:57:04.266909 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.261490    4943 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 12:57:04.266909 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.261495    4943 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 12:57:04.267458 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.261500    4943 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 12:57:04.267458 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.261504    4943 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 12:57:04.267458 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.261508    4943 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 12:57:04.267458 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.261513    4943 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 12:57:04.267458 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.261518    4943 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 12:57:04.267458 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.261522    4943 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 12:57:04.267458 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.261526    4943 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 12:57:04.267458 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.261531    4943 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 12:57:04.267458 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.261536    4943 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 12:57:04.267458 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.261540    4943 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 12:57:04.267458 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.261544    4943 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 12:57:04.267458 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.261549    4943 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 12:57:04.267458 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.261553    4943 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 12:57:04.267458 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.261557    4943 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 12:57:04.267458 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.261562    4943 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 12:57:04.267458 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.261566    4943 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 12:57:04.267458 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.261570    4943 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 12:57:04.267458 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.261574    4943 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 12:57:04.267458 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.261579    4943 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 12:57:04.267458 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.261583    4943 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 12:57:04.268023 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.261587    4943 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 12:57:04.268023 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.261591    4943 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 12:57:04.268023 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.261595    4943 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 12:57:04.268023 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.261602    4943 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 12:57:04.268023 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.261606    4943 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 12:57:04.268023 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.261610    4943 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 12:57:04.268023 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.261614    4943 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 12:57:04.268023 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.261619    4943 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 12:57:04.268023 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.261623    4943 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 12:57:04.268023 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.261627    4943 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 12:57:04.268023 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.261632    4943 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 12:57:04.268023 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.261637    4943 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 12:57:04.268023 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.261642    4943 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 12:57:04.268023 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.261646    4943 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 12:57:04.268023 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.261651    4943 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 12:57:04.268023 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.261655    4943 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 12:57:04.268023 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.261659    4943 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 12:57:04.268023 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.261663    4943 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 12:57:04.268023 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.261670    4943 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 12:57:04.268564 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.261677    4943 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 12:57:04.268564 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.261683    4943 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 12:57:04.268564 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.261688    4943 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 12:57:04.268564 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.261692    4943 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 12:57:04.268564 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.261697    4943 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 12:57:04.268564 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.261701    4943 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 12:57:04.268564 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.261706    4943 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 12:57:04.268564 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.261711    4943 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 12:57:04.268564 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.261715    4943 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 12:57:04.268564 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.261719    4943 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 12:57:04.268564 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.261723    4943 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 12:57:04.268564 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.261728    4943 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 12:57:04.268564 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.261733    4943 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 12:57:04.268564 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.261737    4943 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 12:57:04.268564 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.261741    4943 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 12:57:04.268564 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.261746    4943 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 12:57:04.268564 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.261751    4943 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 12:57:04.268564 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.261756    4943 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 12:57:04.268564 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.261760    4943 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 12:57:04.269103 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.261764    4943 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 12:57:04.269103 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.261768    4943 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 12:57:04.269103 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.261773    4943 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 12:57:04.269103 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.261777    4943 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 12:57:04.269103 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.261782    4943 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 12:57:04.269103 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.261786    4943 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 12:57:04.269103 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.261790    4943 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 12:57:04.269103 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.261795    4943 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 12:57:04.269103 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.261799    4943 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 12:57:04.269103 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.261803    4943 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 12:57:04.269103 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.261808    4943 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 12:57:04.269103 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.261812    4943 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 12:57:04.269103 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.261822    4943 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 12:57:04.269103 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.261827    4943 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 12:57:04.269103 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.261833    4943 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 12:57:04.269103 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.261837    4943 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 12:57:04.269103 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.261841    4943 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 12:57:04.269103 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.261845    4943 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 12:57:04.269103 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.261849    4943 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 12:57:04.269677 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.261853    4943 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 12:57:04.269677 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.261857    4943 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 12:57:04.269677 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.261862    4943 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 12:57:04.269677 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.261867    4943 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 12:57:04.269677 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.262017    4943 flags.go:64] FLAG: --address="0.0.0.0"
Oct 30 12:57:04.269677 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.262029    4943 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Oct 30 12:57:04.269677 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.262046    4943 flags.go:64] FLAG: --anonymous-auth="true"
Oct 30 12:57:04.269677 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.262053    4943 flags.go:64] FLAG: --application-metrics-count-limit="100"
Oct 30 12:57:04.269677 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.262060    4943 flags.go:64] FLAG: --authentication-token-webhook="false"
Oct 30 12:57:04.269677 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.262066    4943 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Oct 30 12:57:04.269677 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.262072    4943 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Oct 30 12:57:04.269677 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.262079    4943 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Oct 30 12:57:04.269677 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.262084    4943 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Oct 30 12:57:04.269677 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.262089    4943 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Oct 30 12:57:04.269677 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.262094    4943 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Oct 30 12:57:04.269677 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.262100    4943 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Oct 30 12:57:04.269677 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.262105    4943 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Oct 30 12:57:04.269677 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.262110    4943 flags.go:64] FLAG: --cgroup-root=""
Oct 30 12:57:04.269677 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.262115    4943 flags.go:64] FLAG: --cgroups-per-qos="true"
Oct 30 12:57:04.269677 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.262120    4943 flags.go:64] FLAG: --client-ca-file=""
Oct 30 12:57:04.269677 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.262125    4943 flags.go:64] FLAG: --cloud-provider="external"
Oct 30 12:57:04.269677 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.262130    4943 flags.go:64] FLAG: --cluster-dns="[]"
Oct 30 12:57:04.269677 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.262137    4943 flags.go:64] FLAG: --cluster-domain=""
Oct 30 12:57:04.269677 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.262142    4943 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Oct 30 12:57:04.270418 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.262147    4943 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Oct 30 12:57:04.270418 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.262155    4943 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Oct 30 12:57:04.270418 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.262160    4943 flags.go:64] FLAG: --container-log-max-files="5"
Oct 30 12:57:04.270418 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.262167    4943 flags.go:64] FLAG: --container-log-max-size="10Mi"
Oct 30 12:57:04.270418 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.262172    4943 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Oct 30 12:57:04.270418 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.262177    4943 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Oct 30 12:57:04.270418 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.262183    4943 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Oct 30 12:57:04.270418 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.262188    4943 flags.go:64] FLAG: --contention-profiling="false"
Oct 30 12:57:04.270418 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.262193    4943 flags.go:64] FLAG: --cpu-cfs-quota="true"
Oct 30 12:57:04.270418 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.262198    4943 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Oct 30 12:57:04.270418 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.262203    4943 flags.go:64] FLAG: --cpu-manager-policy="none"
Oct 30 12:57:04.270418 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.262209    4943 flags.go:64] FLAG: --cpu-manager-policy-options=""
Oct 30 12:57:04.270418 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.262216    4943 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Oct 30 12:57:04.270418 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.262221    4943 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Oct 30 12:57:04.270418 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.262226    4943 flags.go:64] FLAG: --enable-debugging-handlers="true"
Oct 30 12:57:04.270418 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.262231    4943 flags.go:64] FLAG: --enable-load-reader="false"
Oct 30 12:57:04.270418 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.262236    4943 flags.go:64] FLAG: --enable-server="true"
Oct 30 12:57:04.270418 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.262241    4943 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Oct 30 12:57:04.270418 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.262247    4943 flags.go:64] FLAG: --event-burst="100"
Oct 30 12:57:04.270418 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.262252    4943 flags.go:64] FLAG: --event-qps="50"
Oct 30 12:57:04.270418 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.262274    4943 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Oct 30 12:57:04.270418 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.262280    4943 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Oct 30 12:57:04.270418 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.262285    4943 flags.go:64] FLAG: --eviction-hard=""
Oct 30 12:57:04.270418 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.262292    4943 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Oct 30 12:57:04.271092 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.262296    4943 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Oct 30 12:57:04.271092 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.262301    4943 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Oct 30 12:57:04.271092 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.262306    4943 flags.go:64] FLAG: --eviction-soft=""
Oct 30 12:57:04.271092 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.262311    4943 flags.go:64] FLAG: --eviction-soft-grace-period=""
Oct 30 12:57:04.271092 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.262316    4943 flags.go:64] FLAG: --exit-on-lock-contention="false"
Oct 30 12:57:04.271092 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.262320    4943 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Oct 30 12:57:04.271092 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.262326    4943 flags.go:64] FLAG: --experimental-mounter-path=""
Oct 30 12:57:04.271092 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.262331    4943 flags.go:64] FLAG: --fail-cgroupv1="true"
Oct 30 12:57:04.271092 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.262335    4943 flags.go:64] FLAG: --fail-swap-on="true"
Oct 30 12:57:04.271092 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.262343    4943 flags.go:64] FLAG: --feature-gates=""
Oct 30 12:57:04.271092 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.262349    4943 flags.go:64] FLAG: --file-check-frequency="20s"
Oct 30 12:57:04.271092 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.262354    4943 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Oct 30 12:57:04.271092 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.262359    4943 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Oct 30 12:57:04.271092 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.262365    4943 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Oct 30 12:57:04.271092 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.262369    4943 flags.go:64] FLAG: --healthz-port="10248"
Oct 30 12:57:04.271092 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.262374    4943 flags.go:64] FLAG: --help="false"
Oct 30 12:57:04.271092 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.262379    4943 flags.go:64] FLAG: --hostname-override="ip-10-0-142-84.ec2.internal"
Oct 30 12:57:04.271092 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.262384    4943 flags.go:64] FLAG: --housekeeping-interval="10s"
Oct 30 12:57:04.271092 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.262389    4943 flags.go:64] FLAG: --http-check-frequency="20s"
Oct 30 12:57:04.271092 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.262394    4943 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Oct 30 12:57:04.271092 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.262400    4943 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Oct 30 12:57:04.271092 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.262405    4943 flags.go:64] FLAG: --image-gc-high-threshold="85"
Oct 30 12:57:04.271092 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.262410    4943 flags.go:64] FLAG: --image-gc-low-threshold="80"
Oct 30 12:57:04.271092 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.262415    4943 flags.go:64] FLAG: --image-service-endpoint=""
Oct 30 12:57:04.271748 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.262420    4943 flags.go:64] FLAG: --kernel-memcg-notification="false"
Oct 30 12:57:04.271748 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.262425    4943 flags.go:64] FLAG: --kube-api-burst="100"
Oct 30 12:57:04.271748 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.262430    4943 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Oct 30 12:57:04.271748 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.262435    4943 flags.go:64] FLAG: --kube-api-qps="50"
Oct 30 12:57:04.271748 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.262440    4943 flags.go:64] FLAG: --kube-reserved=""
Oct 30 12:57:04.271748 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.262445    4943 flags.go:64] FLAG: --kube-reserved-cgroup=""
Oct 30 12:57:04.271748 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.262449    4943 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Oct 30 12:57:04.271748 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.262454    4943 flags.go:64] FLAG: --kubelet-cgroups=""
Oct 30 12:57:04.271748 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.262459    4943 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Oct 30 12:57:04.271748 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.262463    4943 flags.go:64] FLAG: --lock-file=""
Oct 30 12:57:04.271748 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.262468    4943 flags.go:64] FLAG: --log-cadvisor-usage="false"
Oct 30 12:57:04.271748 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.262473    4943 flags.go:64] FLAG: --log-flush-frequency="5s"
Oct 30 12:57:04.271748 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.262478    4943 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Oct 30 12:57:04.271748 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.262487    4943 flags.go:64] FLAG: --log-json-split-stream="false"
Oct 30 12:57:04.271748 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.262492    4943 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Oct 30 12:57:04.271748 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.262498    4943 flags.go:64] FLAG: --log-text-split-stream="false"
Oct 30 12:57:04.271748 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.262503    4943 flags.go:64] FLAG: --logging-format="text"
Oct 30 12:57:04.271748 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.262511    4943 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Oct 30 12:57:04.271748 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.262517    4943 flags.go:64] FLAG: --make-iptables-util-chains="true"
Oct 30 12:57:04.271748 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.262522    4943 flags.go:64] FLAG: --manifest-url=""
Oct 30 12:57:04.271748 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.262527    4943 flags.go:64] FLAG: --manifest-url-header=""
Oct 30 12:57:04.271748 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.262534    4943 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Oct 30 12:57:04.271748 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.262539    4943 flags.go:64] FLAG: --max-open-files="1000000"
Oct 30 12:57:04.271748 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.262545    4943 flags.go:64] FLAG: --max-pods="110"
Oct 30 12:57:04.271748 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.262550    4943 flags.go:64] FLAG: --maximum-dead-containers="-1"
Oct 30 12:57:04.272412 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.262555    4943 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Oct 30 12:57:04.272412 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.262560    4943 flags.go:64] FLAG: --memory-manager-policy="None"
Oct 30 12:57:04.272412 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.262565    4943 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Oct 30 12:57:04.272412 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.262570    4943 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Oct 30 12:57:04.272412 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.262575    4943 flags.go:64] FLAG: --node-ip="0.0.0.0"
Oct 30 12:57:04.272412 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.262579    4943 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Oct 30 12:57:04.272412 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.262592    4943 flags.go:64] FLAG: --node-status-max-images="50"
Oct 30 12:57:04.272412 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.262597    4943 flags.go:64] FLAG: --node-status-update-frequency="10s"
Oct 30 12:57:04.272412 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.262602    4943 flags.go:64] FLAG: --oom-score-adj="-999"
Oct 30 12:57:04.272412 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.262607    4943 flags.go:64] FLAG: --pod-cidr=""
Oct 30 12:57:04.272412 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.262612    4943 flags.go:64] FLAG: --pod-manifest-path=""
Oct 30 12:57:04.272412 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.262616    4943 flags.go:64] FLAG: --pod-max-pids="-1"
Oct 30 12:57:04.272412 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.262621    4943 flags.go:64] FLAG: --pods-per-core="0"
Oct 30 12:57:04.272412 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.262626    4943 flags.go:64] FLAG: --port="10250"
Oct 30 12:57:04.272412 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.262631    4943 flags.go:64] FLAG: --protect-kernel-defaults="false"
Oct 30 12:57:04.272412 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.262636    4943 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0560abecd3e271ce3"
Oct 30 12:57:04.272412 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.262641    4943 flags.go:64] FLAG: --qos-reserved=""
Oct 30 12:57:04.272412 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.262646    4943 flags.go:64] FLAG: --read-only-port="10255"
Oct 30 12:57:04.272412 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.262651    4943 flags.go:64] FLAG: --register-node="true"
Oct 30 12:57:04.272412 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.262656    4943 flags.go:64] FLAG: --register-with-taints=""
Oct 30 12:57:04.272412 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.262662    4943 flags.go:64] FLAG: --registry-burst="10"
Oct 30 12:57:04.272412 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.262667    4943 flags.go:64] FLAG: --registry-qps="5"
Oct 30 12:57:04.272412 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.262672    4943 flags.go:64] FLAG: --reserved-cpus=""
Oct 30 12:57:04.272412 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.262677    4943 flags.go:64] FLAG: --reserved-memory=""
Oct 30 12:57:04.272412 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.262685    4943 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Oct 30 12:57:04.272412 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.262690    4943 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Oct 30 12:57:04.273051 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.262696    4943 flags.go:64] FLAG: --rotate-certificates="false"
Oct 30 12:57:04.273051 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.262700    4943 flags.go:64] FLAG: --rotate-server-certificates="false"
Oct 30 12:57:04.273051 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.262705    4943 flags.go:64] FLAG: --runonce="false"
Oct 30 12:57:04.273051 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.262710    4943 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Oct 30 12:57:04.273051 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.262716    4943 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Oct 30 12:57:04.273051 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.262721    4943 flags.go:64] FLAG: --seccomp-default="false"
Oct 30 12:57:04.273051 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.262725    4943 flags.go:64] FLAG: --serialize-image-pulls="true"
Oct 30 12:57:04.273051 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.262731    4943 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Oct 30 12:57:04.273051 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.262736    4943 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Oct 30 12:57:04.273051 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.262741    4943 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Oct 30 12:57:04.273051 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.262746    4943 flags.go:64] FLAG: --storage-driver-password="root"
Oct 30 12:57:04.273051 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.262751    4943 flags.go:64] FLAG: --storage-driver-secure="false"
Oct 30 12:57:04.273051 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.262758    4943 flags.go:64] FLAG: --storage-driver-table="stats"
Oct 30 12:57:04.273051 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.262763    4943 flags.go:64] FLAG: --storage-driver-user="root"
Oct 30 12:57:04.273051 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.262767    4943 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Oct 30 12:57:04.273051 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.262773    4943 flags.go:64] FLAG: --sync-frequency="1m0s"
Oct 30 12:57:04.273051 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.262778    4943 flags.go:64] FLAG: --system-cgroups=""
Oct 30 12:57:04.273051 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.262783    4943 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Oct 30 12:57:04.273051 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.262791    4943 flags.go:64] FLAG: --system-reserved-cgroup=""
Oct 30 12:57:04.273051 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.262796    4943 flags.go:64] FLAG: --tls-cert-file=""
Oct 30 12:57:04.273051 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.262800    4943 flags.go:64] FLAG: --tls-cipher-suites="[]"
Oct 30 12:57:04.273051 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.262808    4943 flags.go:64] FLAG: --tls-min-version=""
Oct 30 12:57:04.273051 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.262813    4943 flags.go:64] FLAG: --tls-private-key-file=""
Oct 30 12:57:04.273051 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.262818    4943 flags.go:64] FLAG: --topology-manager-policy="none"
Oct 30 12:57:04.273051 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.262823    4943 flags.go:64] FLAG: --topology-manager-policy-options=""
Oct 30 12:57:04.273731 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.262828    4943 flags.go:64] FLAG: --topology-manager-scope="container"
Oct 30 12:57:04.273731 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.262833    4943 flags.go:64] FLAG: --v="2"
Oct 30 12:57:04.273731 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.262840    4943 flags.go:64] FLAG: --version="false"
Oct 30 12:57:04.273731 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.262846    4943 flags.go:64] FLAG: --vmodule=""
Oct 30 12:57:04.273731 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.262853    4943 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Oct 30 12:57:04.273731 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.262860    4943 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Oct 30 12:57:04.273731 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.263065    4943 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 12:57:04.273731 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.263073    4943 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 12:57:04.273731 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.263078    4943 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 12:57:04.273731 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.263083    4943 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 12:57:04.273731 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.263087    4943 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 12:57:04.273731 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.263092    4943 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 12:57:04.273731 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.263096    4943 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 12:57:04.273731 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.263101    4943 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 12:57:04.273731 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.263106    4943 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 12:57:04.273731 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.263110    4943 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 12:57:04.273731 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.263114    4943 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 12:57:04.273731 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.263119    4943 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 12:57:04.273731 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.263123    4943 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 12:57:04.273731 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.263129    4943 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 12:57:04.273731 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.263134    4943 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 12:57:04.273731 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.263138    4943 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 12:57:04.274418 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.263142    4943 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 12:57:04.274418 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.263147    4943 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 12:57:04.274418 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.263151    4943 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 12:57:04.274418 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.263155    4943 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 12:57:04.274418 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.263160    4943 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 12:57:04.274418 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.263164    4943 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 12:57:04.274418 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.263169    4943 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 12:57:04.274418 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.263173    4943 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 12:57:04.274418 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.263178    4943 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 12:57:04.274418 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.263182    4943 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 12:57:04.274418 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.263187    4943 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 12:57:04.274418 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.263191    4943 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 12:57:04.274418 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.263196    4943 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 12:57:04.274418 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.263200    4943 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 12:57:04.274418 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.263204    4943 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 12:57:04.274418 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.263211    4943 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 12:57:04.274418 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.263216    4943 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 12:57:04.274418 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.263220    4943 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 12:57:04.274418 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.263225    4943 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 12:57:04.274418 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.263230    4943 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 12:57:04.275013 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.263234    4943 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 12:57:04.275013 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.263239    4943 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 12:57:04.275013 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.263243    4943 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 12:57:04.275013 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.263247    4943 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 12:57:04.275013 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.263252    4943 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 12:57:04.275013 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.263271    4943 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 12:57:04.275013 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.263277    4943 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 12:57:04.275013 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.263281    4943 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 12:57:04.275013 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.263285    4943 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 12:57:04.275013 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.263292    4943 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 12:57:04.275013 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.263297    4943 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 12:57:04.275013 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.263301    4943 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 12:57:04.275013 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.263306    4943 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 12:57:04.275013 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.263310    4943 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 12:57:04.275013 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.263314    4943 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 12:57:04.275013 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.263319    4943 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 12:57:04.275013 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.263324    4943 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 12:57:04.275013 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.263328    4943 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 12:57:04.275013 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.263332    4943 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 12:57:04.275580 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.263337    4943 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 12:57:04.275580 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.263342    4943 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 12:57:04.275580 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.263346    4943 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 12:57:04.275580 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.263350    4943 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 12:57:04.275580 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.263355    4943 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 12:57:04.275580 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.263359    4943 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 12:57:04.275580 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.263363    4943 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 12:57:04.275580 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.263367    4943 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 12:57:04.275580 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.263374    4943 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 12:57:04.275580 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.263378    4943 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 12:57:04.275580 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.263384    4943 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 12:57:04.275580 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.263389    4943 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 12:57:04.275580 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.263395    4943 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 12:57:04.275580 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.263401    4943 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 12:57:04.275580 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.263406    4943 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 12:57:04.275580 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.263410    4943 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 12:57:04.275580 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.263415    4943 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 12:57:04.275580 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.263419    4943 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 12:57:04.275580 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.263424    4943 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 12:57:04.276046 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.263428    4943 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 12:57:04.276046 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.263433    4943 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 12:57:04.276046 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.263438    4943 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 12:57:04.276046 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.263444    4943 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 12:57:04.276046 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.263448    4943 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 12:57:04.276046 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.263453    4943 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 12:57:04.276046 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.263457    4943 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 12:57:04.276046 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.263461    4943 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 12:57:04.276046 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.263468    4943 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 12:57:04.276046 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.263475    4943 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 12:57:04.276046 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.263480    4943 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 12:57:04.276046 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.263484    4943 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 12:57:04.276046 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.263488    4943 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 12:57:04.276046 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.263493    4943 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 12:57:04.276046 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.263497    4943 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 12:57:04.276046 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.263502    4943 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 12:57:04.276046 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.263506    4943 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 12:57:04.276046 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.263510    4943 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 12:57:04.276046 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.263515    4943 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 12:57:04.276742 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.263519    4943 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 12:57:04.276742 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.263524    4943 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 12:57:04.276742 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.263530    4943 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 12:57:04.276742 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.263534    4943 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 12:57:04.276742 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.263538    4943 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 12:57:04.276742 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.263543    4943 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 12:57:04.276742 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.263548    4943 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 12:57:04.276742 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.263554    4943 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 12:57:04.276742 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.263559    4943 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 12:57:04.276742 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.263565    4943 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 12:57:04.276742 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.263570    4943 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 12:57:04.276742 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.263575    4943 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 12:57:04.276742 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.263579    4943 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 12:57:04.276742 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.263623    4943 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 30 12:57:04.276742 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.263678    4943 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 12:57:04.276742 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.263684    4943 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 12:57:04.277216 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.263690    4943 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 12:57:04.277216 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.263697    4943 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 12:57:04.277216 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.263703    4943 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 12:57:04.277216 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.263708    4943 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 12:57:04.277216 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.263712    4943 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 12:57:04.277216 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.263717    4943 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 12:57:04.277216 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.263721    4943 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 12:57:04.277216 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.263725    4943 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 12:57:04.277216 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.263730    4943 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 12:57:04.277216 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.263734    4943 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 12:57:04.277216 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.263739    4943 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 12:57:04.277216 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.263743    4943 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 12:57:04.277216 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.263748    4943 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 12:57:04.277216 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.263752    4943 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 12:57:04.277216 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.263757    4943 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 12:57:04.277216 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.263761    4943 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 12:57:04.277216 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.263766    4943 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 12:57:04.277216 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.263770    4943 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 12:57:04.277216 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.263776    4943 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 12:57:04.277216 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.263781    4943 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 12:57:04.277773 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.263785    4943 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 12:57:04.277773 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.263790    4943 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 12:57:04.277773 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.263795    4943 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 12:57:04.277773 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.263799    4943 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 12:57:04.277773 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.263804    4943 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 12:57:04.277773 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.263808    4943 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 12:57:04.277773 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.263812    4943 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 12:57:04.277773 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.263816    4943 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 12:57:04.277773 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.263821    4943 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 12:57:04.277773 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.263825    4943 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 12:57:04.277773 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.263829    4943 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 12:57:04.277773 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.263833    4943 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 12:57:04.277773 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.263837    4943 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 12:57:04.277773 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.263842    4943 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 12:57:04.277773 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.263847    4943 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 12:57:04.277773 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.263852    4943 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 12:57:04.277773 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.263856    4943 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 12:57:04.277773 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.263861    4943 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 12:57:04.277773 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.263865    4943 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 12:57:04.277773 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.263870    4943 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 12:57:04.278381 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.263875    4943 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 12:57:04.278381 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.263880    4943 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 12:57:04.278381 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.263884    4943 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 12:57:04.278381 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.263888    4943 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 12:57:04.278381 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.263893    4943 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 12:57:04.278381 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.263897    4943 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 12:57:04.278381 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.263902    4943 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 12:57:04.278381 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.263906    4943 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 12:57:04.278381 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.263911    4943 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 12:57:04.278381 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.263915    4943 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 12:57:04.278381 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.263920    4943 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 12:57:04.278381 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.263924    4943 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 12:57:04.278381 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.263928    4943 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 12:57:04.278381 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.263933    4943 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 12:57:04.278381 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.263937    4943 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 12:57:04.278381 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.263942    4943 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 12:57:04.278381 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.263947    4943 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 12:57:04.278381 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.263952    4943 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 12:57:04.278381 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.263957    4943 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 12:57:04.278381 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.263961    4943 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 12:57:04.278866 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.263965    4943 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 12:57:04.278866 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.263970    4943 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 12:57:04.278866 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.263974    4943 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 12:57:04.278866 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.263979    4943 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 12:57:04.278866 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.263983    4943 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 12:57:04.278866 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.263989    4943 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 12:57:04.278866 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.263994    4943 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 12:57:04.278866 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.263998    4943 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 12:57:04.278866 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.264003    4943 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 12:57:04.278866 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.264008    4943 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 12:57:04.278866 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.264012    4943 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 12:57:04.278866 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.264016    4943 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 12:57:04.278866 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.264020    4943 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 12:57:04.278866 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.264025    4943 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 12:57:04.278866 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.264029    4943 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 12:57:04.278866 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.264034    4943 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 12:57:04.278866 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.264038    4943 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 12:57:04.278866 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.264046    4943 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 12:57:04.278866 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.264052    4943 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 12:57:04.279350 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.264057    4943 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 12:57:04.279350 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.264062    4943 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 12:57:04.279350 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.264066    4943 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 12:57:04.279350 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.264071    4943 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 12:57:04.279350 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.264075    4943 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 12:57:04.279350 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.264080    4943 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 12:57:04.279350 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.264085    4943 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 12:57:04.279350 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.264089    4943 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 12:57:04.279350 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.264094    4943 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 12:57:04.279350 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.264098    4943 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 12:57:04.279350 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.264103    4943 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 12:57:04.279350 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.264107    4943 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 12:57:04.279350 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.264112    4943 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 12:57:04.279350 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.264119    4943 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 12:57:04.279350 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.264125    4943 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 12:57:04.279350 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.264130    4943 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 12:57:04.279350 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.264135    4943 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 12:57:04.279350 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.264140    4943 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 12:57:04.279350 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.264146    4943 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 12:57:04.279816 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.264150    4943 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 12:57:04.279816 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.264155    4943 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 12:57:04.279816 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.264159    4943 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 12:57:04.279816 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.264164    4943 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 12:57:04.279816 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.264168    4943 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 12:57:04.279816 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.264173    4943 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 12:57:04.279816 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.269309    4943 server.go:525] "Kubelet version" kubeletVersion="v1.35.9"
Oct 30 12:57:04.279816 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.269325    4943 server.go:527] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Oct 30 12:57:04.279816 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.269345    4943 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Oct 30 12:57:04.279816 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.269350    4943 watchdog_linux.go:138] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Oct 30 12:57:04.279816 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.269419    4943 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 12:57:04.279816 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.269424    4943 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 12:57:04.279816 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.269427    4943 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 12:57:04.279816 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.269431    4943 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 12:57:04.279816 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.269434    4943 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 12:57:04.279816 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.269437    4943 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 12:57:04.279816 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.269441    4943 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 12:57:04.279816 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.269444    4943 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 12:57:04.279816 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.269447    4943 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 12:57:04.280336 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.269450    4943 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 12:57:04.280336 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.269452    4943 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 12:57:04.280336 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.269456    4943 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 12:57:04.280336 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.269459    4943 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 12:57:04.280336 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.269462    4943 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 12:57:04.280336 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.269465    4943 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 12:57:04.280336 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.269469    4943 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 12:57:04.280336 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.269471    4943 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 12:57:04.280336 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.269475    4943 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 12:57:04.280336 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.269477    4943 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 12:57:04.280336 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.269480    4943 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 12:57:04.280336 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.269483    4943 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 12:57:04.280336 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.269486    4943 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 12:57:04.280336 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.269489    4943 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 12:57:04.280336 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.269491    4943 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 12:57:04.280336 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.269494    4943 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 12:57:04.280336 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.269497    4943 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 12:57:04.280336 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.269501    4943 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 12:57:04.280336 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.269503    4943 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 12:57:04.280336 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.269506    4943 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 12:57:04.280844 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.269509    4943 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 12:57:04.280844 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.269512    4943 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 12:57:04.280844 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.269516    4943 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 12:57:04.280844 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.269518    4943 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 12:57:04.280844 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.269521    4943 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 12:57:04.280844 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.269524    4943 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 12:57:04.280844 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.269527    4943 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 12:57:04.280844 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.269530    4943 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 12:57:04.280844 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.269532    4943 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 12:57:04.280844 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.269535    4943 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 12:57:04.280844 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.269538    4943 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 12:57:04.280844 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.269541    4943 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 12:57:04.280844 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.269544    4943 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 12:57:04.280844 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.269547    4943 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 12:57:04.280844 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.269550    4943 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 12:57:04.280844 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.269553    4943 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 12:57:04.280844 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.269555    4943 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 12:57:04.280844 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.269559    4943 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 12:57:04.280844 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.269561    4943 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 12:57:04.280844 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.269564    4943 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 12:57:04.281346 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.269567    4943 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 12:57:04.281346 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.269570    4943 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 12:57:04.281346 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.269573    4943 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 12:57:04.281346 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.269575    4943 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 12:57:04.281346 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.269578    4943 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 12:57:04.281346 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.269581    4943 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 12:57:04.281346 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.269584    4943 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 12:57:04.281346 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.269587    4943 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 12:57:04.281346 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.269589    4943 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 12:57:04.281346 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.269593    4943 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 12:57:04.281346 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.269596    4943 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 12:57:04.281346 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.269598    4943 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 12:57:04.281346 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.269601    4943 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 12:57:04.281346 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.269604    4943 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 12:57:04.281346 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.269607    4943 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 12:57:04.281346 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.269610    4943 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 12:57:04.281346 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.269613    4943 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 12:57:04.281346 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.269616    4943 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 12:57:04.281346 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.269618    4943 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 12:57:04.281816 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.269621    4943 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 12:57:04.281816 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.269624    4943 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 12:57:04.281816 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.269627    4943 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 12:57:04.281816 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.269629    4943 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 12:57:04.281816 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.269632    4943 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 12:57:04.281816 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.269635    4943 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 12:57:04.281816 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.269637    4943 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 12:57:04.281816 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.269640    4943 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 12:57:04.281816 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.269643    4943 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 12:57:04.281816 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.269646    4943 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 12:57:04.281816 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.269649    4943 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 12:57:04.281816 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.269653    4943 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 12:57:04.281816 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.269656    4943 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 12:57:04.281816 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.269658    4943 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 12:57:04.281816 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.269661    4943 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 12:57:04.281816 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.269664    4943 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 12:57:04.281816 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.269666    4943 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 12:57:04.281816 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.269669    4943 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 12:57:04.281816 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.269672    4943 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 12:57:04.282355 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.269675    4943 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 12:57:04.282355 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.269679    4943 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 12:57:04.282355 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.269684    4943 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 12:57:04.282355 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.269688    4943 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 12:57:04.282355 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.269691    4943 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 12:57:04.282355 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.269694    4943 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 12:57:04.282355 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.269697    4943 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 12:57:04.282355 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.269700    4943 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 12:57:04.282355 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.269703    4943 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 12:57:04.282355 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.269706    4943 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 12:57:04.282355 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.269708    4943 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 12:57:04.282355 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.269711    4943 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 12:57:04.282355 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.269714    4943 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 12:57:04.282355 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.269716    4943 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 12:57:04.282355 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.269719    4943 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 12:57:04.282355 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.269724    4943 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 12:57:04.282355 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.269727    4943 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 12:57:04.282355 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.269731    4943 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 12:57:04.282355 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.269734    4943 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 12:57:04.283044 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.269764    4943 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 30 12:57:04.283044 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.269927    4943 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 12:57:04.283044 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.269933    4943 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 12:57:04.283044 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.269936    4943 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 12:57:04.283044 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.269939    4943 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 12:57:04.283044 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.269942    4943 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 12:57:04.283044 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.269945    4943 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 12:57:04.283044 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.269948    4943 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 12:57:04.283044 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.269951    4943 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 12:57:04.283044 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.269954    4943 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 12:57:04.283044 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.269957    4943 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 12:57:04.283044 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.269960    4943 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 12:57:04.283044 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.269962    4943 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 12:57:04.283044 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.269965    4943 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 12:57:04.283044 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.269968    4943 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 12:57:04.283044 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.269971    4943 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 12:57:04.283694 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.269974    4943 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 12:57:04.283694 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.269977    4943 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 12:57:04.283694 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.269979    4943 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 12:57:04.283694 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.269982    4943 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 12:57:04.283694 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.269985    4943 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 12:57:04.283694 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.269988    4943 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 12:57:04.283694 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.269991    4943 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 12:57:04.283694 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.269993    4943 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 12:57:04.283694 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.269996    4943 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 12:57:04.283694 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.269999    4943 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 12:57:04.283694 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.270002    4943 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 12:57:04.283694 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.270005    4943 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 12:57:04.283694 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.270008    4943 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 12:57:04.283694 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.270010    4943 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 12:57:04.283694 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.270013    4943 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 12:57:04.283694 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.270016    4943 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 12:57:04.283694 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.270019    4943 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 12:57:04.283694 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.270021    4943 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 12:57:04.283694 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.270025    4943 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 12:57:04.284174 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.270028    4943 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 12:57:04.284174 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.270030    4943 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 12:57:04.284174 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.270033    4943 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 12:57:04.284174 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.270036    4943 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 12:57:04.284174 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.270040    4943 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 12:57:04.284174 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.270042    4943 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 12:57:04.284174 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.270045    4943 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 12:57:04.284174 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.270048    4943 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 12:57:04.284174 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.270051    4943 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 12:57:04.284174 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.270053    4943 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 12:57:04.284174 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.270056    4943 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 12:57:04.284174 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.270059    4943 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 12:57:04.284174 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.270061    4943 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 12:57:04.284174 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.270064    4943 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 12:57:04.284174 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.270067    4943 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 12:57:04.284174 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.270069    4943 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 12:57:04.284174 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.270072    4943 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 12:57:04.284174 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.270075    4943 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 12:57:04.284174 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.270078    4943 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 12:57:04.284703 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.270080    4943 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 12:57:04.284703 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.270083    4943 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 12:57:04.284703 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.270086    4943 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 12:57:04.284703 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.270089    4943 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 12:57:04.284703 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.270094    4943 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 12:57:04.284703 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.270097    4943 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 12:57:04.284703 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.270101    4943 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 12:57:04.284703 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.270105    4943 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 12:57:04.284703 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.270108    4943 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 12:57:04.284703 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.270111    4943 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 12:57:04.284703 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.270117    4943 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 12:57:04.284703 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.270120    4943 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 12:57:04.284703 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.270123    4943 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 12:57:04.284703 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.270125    4943 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 12:57:04.284703 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.270128    4943 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 12:57:04.284703 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.270131    4943 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 12:57:04.284703 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.270134    4943 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 12:57:04.284703 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.270137    4943 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 12:57:04.284703 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.270140    4943 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 12:57:04.285168 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.270143    4943 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 12:57:04.285168 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.270145    4943 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 12:57:04.285168 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.270148    4943 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 12:57:04.285168 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.270150    4943 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 12:57:04.285168 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.270153    4943 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 12:57:04.285168 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.270156    4943 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 12:57:04.285168 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.270159    4943 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 12:57:04.285168 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.270162    4943 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 12:57:04.285168 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.270164    4943 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 12:57:04.285168 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.270167    4943 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 12:57:04.285168 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.270170    4943 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 12:57:04.285168 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.270172    4943 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 12:57:04.285168 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.270175    4943 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 12:57:04.285168 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.270178    4943 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 12:57:04.285168 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.270181    4943 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 12:57:04.285168 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.270184    4943 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 12:57:04.285168 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.270186    4943 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 12:57:04.285168 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.270189    4943 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 12:57:04.285168 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.270192    4943 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 12:57:04.285168 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.270195    4943 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 12:57:04.285727 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.270198    4943 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 12:57:04.285727 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.270200    4943 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 12:57:04.285727 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.270203    4943 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 12:57:04.285727 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.270208    4943 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 12:57:04.285727 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.270211    4943 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 12:57:04.285727 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.270214    4943 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 12:57:04.285727 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.270217    4943 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 12:57:04.285727 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.270219    4943 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 12:57:04.285727 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.270222    4943 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 12:57:04.285727 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.270225    4943 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 12:57:04.285727 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.270227    4943 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 12:57:04.285727 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.270230    4943 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 12:57:04.285727 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.270234    4943 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 12:57:04.285727 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.270236    4943 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 12:57:04.285727 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.270272    4943 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 30 12:57:04.285727 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.270309    4943 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 12:57:04.286114 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.270313    4943 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 12:57:04.286114 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.270316    4943 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 12:57:04.286114 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.270319    4943 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 12:57:04.286114 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.270322    4943 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 12:57:04.286114 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.270325    4943 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 12:57:04.286114 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.270328    4943 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 12:57:04.286114 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.270331    4943 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 12:57:04.286114 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.270334    4943 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 12:57:04.286114 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.270336    4943 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 12:57:04.286114 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.270339    4943 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 12:57:04.286114 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.270342    4943 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 12:57:04.286114 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.270345    4943 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 12:57:04.286114 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.270347    4943 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 12:57:04.286114 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.270350    4943 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 12:57:04.286114 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.270353    4943 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 12:57:04.286114 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.270356    4943 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 12:57:04.286114 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.270358    4943 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 12:57:04.286114 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.270361    4943 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 12:57:04.286114 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.270364    4943 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 12:57:04.286616 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.270368    4943 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 12:57:04.286616 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.270371    4943 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 12:57:04.286616 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.270373    4943 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 12:57:04.286616 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.270376    4943 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 12:57:04.286616 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.270379    4943 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 12:57:04.286616 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.270382    4943 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 12:57:04.286616 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.270385    4943 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 12:57:04.286616 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.270387    4943 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 12:57:04.286616 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.270390    4943 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 12:57:04.286616 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.270393    4943 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 12:57:04.286616 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.270396    4943 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 12:57:04.286616 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.270399    4943 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 12:57:04.286616 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.270402    4943 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 12:57:04.286616 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.270404    4943 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 12:57:04.286616 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.270408    4943 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 12:57:04.286616 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.270412    4943 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 12:57:04.286616 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.270415    4943 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 12:57:04.286616 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.270420    4943 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 12:57:04.286616 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.270424    4943 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 12:57:04.287075 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.270428    4943 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 12:57:04.287075 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.270431    4943 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 12:57:04.287075 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.270434    4943 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 12:57:04.287075 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.270437    4943 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 12:57:04.287075 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.270440    4943 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 12:57:04.287075 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.270443    4943 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 12:57:04.287075 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.270446    4943 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 12:57:04.287075 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.270448    4943 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 12:57:04.287075 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.270451    4943 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 12:57:04.287075 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.270454    4943 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 12:57:04.287075 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.270457    4943 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 12:57:04.287075 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.270460    4943 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 12:57:04.287075 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.270462    4943 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 12:57:04.287075 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.270467    4943 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 12:57:04.287075 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.270470    4943 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 12:57:04.287075 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.270473    4943 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 12:57:04.287075 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.270477    4943 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 12:57:04.287075 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.270480    4943 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 12:57:04.287075 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.270482    4943 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 12:57:04.287075 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.270485    4943 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 12:57:04.287581 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.270488    4943 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 12:57:04.287581 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.270491    4943 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 12:57:04.287581 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.270493    4943 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 12:57:04.287581 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.270496    4943 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 12:57:04.287581 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.270499    4943 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 12:57:04.287581 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.270502    4943 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 12:57:04.287581 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.270505    4943 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 12:57:04.287581 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.270508    4943 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 12:57:04.287581 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.270511    4943 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 12:57:04.287581 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.270513    4943 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 12:57:04.287581 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.270517    4943 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 12:57:04.287581 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.270520    4943 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 12:57:04.287581 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.270523    4943 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 12:57:04.287581 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.270525    4943 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 12:57:04.287581 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.270528    4943 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 12:57:04.287581 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.270531    4943 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 12:57:04.287581 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.270533    4943 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 12:57:04.287581 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.270536    4943 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 12:57:04.287581 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.270539    4943 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 12:57:04.287581 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.270541    4943 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 12:57:04.288095 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.270544    4943 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 12:57:04.288095 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.270547    4943 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 12:57:04.288095 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.270550    4943 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 12:57:04.288095 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.270553    4943 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 12:57:04.288095 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.270555    4943 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 12:57:04.288095 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.270560    4943 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 12:57:04.288095 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.270563    4943 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 12:57:04.288095 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.270566    4943 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 12:57:04.288095 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.270570    4943 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 12:57:04.288095 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.270573    4943 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 12:57:04.288095 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.270576    4943 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 12:57:04.288095 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.270579    4943 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 12:57:04.288095 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.270582    4943 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 12:57:04.288095 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.270585    4943 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 12:57:04.288095 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.270588    4943 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 12:57:04.288095 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.270590    4943 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 12:57:04.288095 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.270593    4943 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 12:57:04.288095 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.270597    4943 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 12:57:04.288095 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.270600    4943 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 12:57:04.288625 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.270602    4943 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 12:57:04.288625 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.270605    4943 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 12:57:04.288625 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.270608    4943 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 12:57:04.288625 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.270611    4943 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 12:57:04.288625 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.270613    4943 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 12:57:04.288625 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.270616    4943 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 12:57:04.288625 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.270620    4943 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 12:57:04.288625 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:04.270623    4943 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 12:57:04.288625 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.270688    4943 server.go:957] "Client rotation is on, will bootstrap in background"
Oct 30 12:57:04.288625 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.271638    4943 bootstrap.go:85] "Current kubeconfig file contents are still valid, no bootstrap necessary"
Oct 30 12:57:04.288625 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.271691    4943 certificate_store.go:147] "Loading cert/key pair from a file" filePath="/var/lib/kubelet/pki/kubelet-client-current.pem"
Oct 30 12:57:04.288625 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.271864    4943 server.go:1014] "Starting client certificate rotation"
Oct 30 12:57:04.288625 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.271997    4943 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Oct 30 12:57:04.288625 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.272042    4943 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-10-29 12:51:23 +0000 UTC" deadline="2028-04-11 06:55:11.123815776 +0000 UTC"
Oct 30 12:57:04.288625 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.272081    4943 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12689h58m6.851736422s"
Oct 30 12:57:04.288625 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.272362    4943 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Oct 30 12:57:04.289079 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.273483    4943 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Oct 30 12:57:04.289079 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.274792    4943 log.go:25] "Validated CRI v1 runtime API"
Oct 30 12:57:04.289079 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.276522    4943 log.go:25] "Validated CRI v1 image API"
Oct 30 12:57:04.289079 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.277210    4943 server.go:1424] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Oct 30 12:57:04.289079 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.278958    4943 fs.go:135] Filesystem UUIDs: map[65651b62-86c7-4011-96cd-b774c13a8db5:/dev/nvme0n1p4 70c3b896-a70f-441a-aee0-27cfdf2fa572:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2]
Oct 30 12:57:04.289215 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.278980    4943 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:24 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:26 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/08239c831d96bfc6cf0bec4a44ccf8911e596dd61d29ac6eacc10da29b2e6b5e/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/08239c831d96bfc6cf0bec4a44ccf8911e596dd61d29ac6eacc10da29b2e6b5e/userdata/shm major:0 minor:218 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/15f0ea4b68ee48208edbca8d812819c1cfee317092c13305aa22c767e5ce6480/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/15f0ea4b68ee48208edbca8d812819c1cfee317092c13305aa22c767e5ce6480/userdata/shm major:0 minor:232 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/30f12e40f554fc638dc898f183ece14727f45446ed0a5f886f254f2034e43302/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/30f12e40f554fc638dc898f183ece14727f45446ed0a5f886f254f2034e43302/userdata/shm major:0 minor:340 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/3389cdf237dd6b19ba280bf5738bd8a7a90c3f3d03e56b7ea402b586ead69d51/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/3389cdf237dd6b19ba280bf5738bd8a7a90c3f3d03e56b7ea402b586ead69d51/userdata/shm major:0 minor:294 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/4ca8262c580200ac1c52c3d3783741b6914f7de30bbb7656c13e92eb6b55620e/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/4ca8262c580200ac1c52c3d3783741b6914f7de30bbb7656c13e92eb6b55620e/userdata/shm major:0 minor:72 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/542a1406489aeed500a43e313ecb063b343fb53aa6b064ac09cbc4346db8987c/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/542a1406489aeed500a43e313ecb063b343fb53aa6b064ac09cbc4346db8987c/userdata/shm major:0 minor:94 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/56df6b59efaeea83af00f4daa02105b051a8b360001d76963a79837c1727270e/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/56df6b59efaeea83af00f4daa02105b051a8b360001d76963a79837c1727270e/userdata/shm major:0 minor:48 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/59fa8cb30606c434c1df30aa0f941a6c6d80de50607501c039b61cfc0a8f5759/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/59fa8cb30606c434c1df30aa0f941a6c6d80de50607501c039b61cfc0a8f5759/userdata/shm major:0 minor:216 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/72c23193bad92438cb473b27beb7f3f98a9a10064ec52310d78c6637bae06e28/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/72c23193bad92438cb473b27beb7f3f98a9a10064ec52310d78c6637bae06e28/userdata/shm major:0 minor:79 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/78bd4381c30eb75c22521d05e237a70dfabe47221dacc0c74ebcbd5e36ff8fbf/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/78bd4381c30eb75c22521d05e237a70dfabe47221dacc0c74ebcbd5e36ff8fbf/userdata/shm major:0 minor:296 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/7950ed323f5ca09de76caa701be569bfe27144ce3c8f0be537a097a4d2ec2e47/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/7950ed323f5ca09de76caa701be569bfe27144ce3c8f0be537a097a4d2ec2e47/userdata/shm major:0 minor:76 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/842ea7f7aa13c414c50b554ba5482c42c5614c8418f34fd986e33d90be9e34d0/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/842ea7f7aa13c414c50b554ba5482c42c5614c8418f34fd986e33d90be9e34d0/userdata/shm major:0 minor:68 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/8a35b228406b2d0201049df742444e4ce6aceda6624f683a21b2f712acc51808/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/8a35b228406b2d0201049df742444e4ce6aceda6624f683a21b2f712acc51808/userdata/shm major:0 minor:85 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/91591585f74679f5a203056eee04a9ff065c4244679fef41594168eba3ad7111/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/91591585f74679f5a203056eee04a9ff065c4244679fef41594168eba3ad7111/userdata/shm major:0 minor:220 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/921c5569e0b7dd8f4c9893322f95435bbff7ce00371964f243fd87193d22af85/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/921c5569e0b7dd8f4c9893322f95435bbff7ce00371964f243fd87193d22af85/userdata/shm major:0 minor:95 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/9387d634b449ee1cf65146c83f1fd3d16ba72de247544e9952f79af081174b7f/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/9387d634b449ee1cf65146c83f1fd3d16ba72de247544e9952f79af081174b7f/userdata/shm major:0 minor:236 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/afc6d32b630af29555dbee82da400929a106c898110d62a6e248dac42fa99dd1/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/afc6d32b630af29555dbee82da400929a106c898110d62a6e248dac42fa99dd1/userdata/shm major:0 minor:47 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/c501eda152de5cd4a5dedb5febbd60de386dfd9fd8cc677a2ebf5afebd205cad/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/c501eda152de5cd4a5dedb5febbd60de386dfd9fd8cc677a2ebf5afebd205cad/userdata/shm major:0 minor:223 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/d32290f4009892eecbff7fd69950ecfd46024944d1fb20ad7609255ba446c867/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/d32290f4009892eecbff7fd69950ecfd46024944d1fb20ad7609255ba446c867/userdata/shm major:0 minor:77 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/d8be95465e29616c512fecb557b5599168658616bffa156020d86ab0a30942f4/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/d8be95465e29616c512fecb557b5599168658616bffa156020d86ab0a30942f4/userdata/shm major:0 minor:214 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/e463ce667e1fda351b2fee3fbb5b5a5194516d6c0afc229100c0f8501e19fe57/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/e463ce667e1fda351b2fee3fbb5b5a5194516d6c0afc229100c0f8501e19fe57/userdata/shm major:0 minor:246 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/ef64188e81684686678c54e0cb2634c8321a9f8e65abacee44523832bc1b9c45/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/ef64188e81684686678c54e0cb2634c8321a9f8e65abacee44523832bc1b9c45/userdata/shm major:0 minor:242 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/f64856dcc52a80712efd8546500b0aa1dcf1f82708d8fc6b33eca1a2baa4320c/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/f64856dcc52a80712efd8546500b0aa1dcf1f82708d8fc6b33eca1a2baa4320c/userdata/shm major:0 minor:244 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:32 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/0905782c-26ad-4dc8-93cf-d6f211ea7af5/volumes/kubernetes.io~projected/kube-api-access-swxqr:{mountpoint:/var/lib/kubelet/pods/0905782c-26ad-4dc8-93cf-d6f211ea7af5/volumes/kubernetes.io~projected/kube-api-access-swxqr major:0 minor:197 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/16402666-f516-49d1-be0d-f6ab7234e637/volumes/kubernetes.io~projected/kube-api-access-8xmnf:{mountpoint:/var/lib/kubelet/pods/16402666-f516-49d1-be0d-f6ab7234e637/volumes/kubernetes.io~projected/kube-api-access-8xmnf major:0 minor:62 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/19b38c46-bfd4-4ed0-a764-f7b672f18cc6/volumes/kubernetes.io~projected/kube-api-access-qwmfv:{mountpoint:/var/lib/kubelet/pods/19b38c46-bfd4-4ed0-a764-f7b672f18cc6/volumes/kubernetes.io~projected/kube-api-access-qwmfv major:0 minor:212 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/1c405568-cb88-4485-ae2c-3614cc31ced6/volumes/kubernetes.io~projected/bound-sa-token:{mountpoint:/var/lib/kubelet/pods/1c405568-cb88-4485-ae2c-3614cc31ced6/volumes/kubernetes.io~projected/bound-sa-token major:0 minor:233 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/1c405568-cb88-4485-ae2c-3614cc31ced6/volumes/kubernetes.io~projected/kube-api-access-54kft:{mountpoint:/var/lib/kubelet/pods/1c405568-cb88-4485-ae2c-3614cc31ced6/volumes/kubernetes.io~projected/kube-api-access-54kft major:0 minor:230 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/1c405568-cb88-4485-ae2c-3614cc31ced6/volumes/kubernetes.io~secret/image-registry-private-configuration:{mountpoint:/var/lib/kubelet/pods/1c405568-cb88-4485-ae2c-3614cc31ced6/volumes/kubernetes.io~secret/image-registry-private-configuration major:0 minor:227 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/1c405568-cb88-4485-ae2c-3614cc31ced6/volumes/kubernetes.io~secret/installation-pull-secrets:{mountpoint:/var/lib/kubelet/pods/1c405568-cb88-4485-ae2c-3614cc31ced6/volumes/kubernetes.io~secret/installation-pull-secrets major:0 minor:228 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/23297e74-48f9-47b7-bb5a-80189bdbf2c4/volumes/kubernetes.io~projected/kube-api-access-jz42t:{mountpoint:/var/lib/kubelet/pods/23297e74-48f9-47b7-bb5a-80189bdbf2c4/volumes/kubernetes.io~projected/kube-api-access-jz42t major:0 minor:229 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/27c55f84-4e9a-44de-94ea-02258d6bc747/volumes/kubernetes.io~projected/kube-api-access-hfnqv:{mountpoint:/var/lib/kubelet/pods/27c55f84-4e9a-44de-94ea-02258d6bc747/volumes/kubernetes.io~projected/kube-api-access-hfnqv major:0 minor:211 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/27c55f84-4e9a-44de-94ea-02258d6bc747/volumes/kubernetes.io~secret/serving-cert:{mountpoint:/var/lib/kubelet/pods/27c55f84-4e9a-44de-94ea-02258d6bc747/volumes/kubernetes.io~secret/serving-cert major:0 minor:205 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/301eebcb-3d48-4eef-a0de-a1173e23ad65/volumes/kubernetes.io~projected/kube-api-access-49z5c:{mountpoint:/var/lib/kubelet/pods/301eebcb-3d48-4eef-a0de-a1173e23ad65/volumes/kubernetes.io~projected/kube-api-access-49z5c major:0 minor:201 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/4817ec63-f4d5-4420-9417-148346ff8bb5/volume-subpaths/run-systemd/ovnkube-controller/6:{mountpoint:/var/lib/kubelet/pods/4817ec63-f4d5-4420-9417-148346ff8bb5/volume-subpaths/run-systemd/ovnkube-controller/6 major:0 minor:26 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/4817ec63-f4d5-4420-9417-148346ff8bb5/volumes/kubernetes.io~projected/kube-api-access-7rwfx:{mountpoint:/var/lib/kubelet/pods/4817ec63-f4d5-4420-9417-148346ff8bb5/volumes/kubernetes.io~projected/kube-api-access-7rwfx major:0 minor:59 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/4817ec63-f4d5-4420-9417-148346ff8bb5/volumes/kubernetes.io~secret/ovn-node-metrics-cert:{mountpoint:/var/lib/kubelet/pods/4817ec63-f4d5-4420-9417-148346ff8bb5/volumes/kubernetes.io~secret/ovn-node-metrics-cert major:0 minor:56 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/51d01732-87a4-42b0-a6d5-a3fac689d7f6/volumes/kubernetes.io~secret/agent-certs:{mountpoint:/var/lib/kubelet/pods/51d01732-87a4-42b0-a6d5-a3fac689d7f6/volumes/kubernetes.io~secret/agent-certs major:0 minor:58 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/55fa4842-3eb4-4e52-a4fc-18876ca409d5/volumes/kubernetes.io~projected/kube-api-access-zg85t:{mountpoint:/var/lib/kubelet/pods/55fa4842-3eb4-4e52-a4fc-18876ca409d5/volumes/kubernetes.io~projected/kube-api-access-zg85t major:0 minor:312 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/56ff7ccd-573e-4bb2-a2b7-199a0346ed9f/volumes/kubernetes.io~empty-dir/etc-tuned:{mountpoint:/var/lib/kubelet/pods/56ff7ccd-573e-4bb2-a2b7-199a0346ed9f/volumes/kubernetes.io~empty-dir/etc-tuned major:0 minor:57 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/56ff7ccd-573e-4bb2-a2b7-199a0346ed9f/volumes/kubernetes.io~empty-dir/tmp:{mountpoint:/var/lib/kubelet/pods/56ff7ccd-573e-4bb2-a2b7-199a0346ed9f/volumes/kubernetes.io~empty-dir/tmp major:0 minor:55 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/56ff7ccd-573e-4bb2-a2b7-199a0346ed9f/volumes/kubernetes.io~projected/kube-api-access-66dlr:{mountpoint:/var/lib/kubelet/pods/56ff7ccd-573e-4bb2-a2b7-199a0346ed9f/volumes/kubernetes.io~projected/kube-api-access-66dlr major:0 minor:64 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/7ed65395-2cbd-46bf-81dc-fe307f3a9345/volumes/kubernetes.io~projected/kube-api-access-gz9d6:{mountpoint:/var/lib/kubelet/pods/7ed65395-2cbd-46bf-81dc-fe307f3a9345/volumes/kubernetes.io~projected/kube-api-access-gz9d6 major:0 minor:240 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/85f4c399-0089-4399-86e2-069834f89286/volumes/kubernetes.io~projected/kube-api-access-rls7j:{mountpoint:/var/lib/kubelet/pods/85f4c399-0089-4399-86e2-069834f89286/volumes/kubernetes.io~projected/kube-api-access-rls7j major:0 minor:210 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/85f4c399-0089-4399-86e2-069834f89286/volumes/kubernetes.io~secret/serving-cert:{mountpoint:/var/lib/kubelet/pods/85f4c399-0089-4399-86e2-069834f89286/volumes/kubernetes.io~secret/serving-cert major:0 minor:203 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/8d767e44-14df-4ea6-b391-bdf265773535/volumes/kubernetes.io~projected/kube-api-access-9vr7r:{mountpoint:/var/lib/kubelet/pods/8d767e44-14df-4ea6-b391-bdf265773535/volumes/kubernetes.io~projected/kube-api-access-9vr7r major:0 minor:288 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/98e7e1f3-2ca6-45de-a5d8-d27c22e257b1/volumes/kubernetes.io~projected/kube-api-access-hbsbr:{mountpoint:/var/lib/kubelet/pods/98e7e1f3-2ca6-45de-a5d8-d27c22e257b1/volumes/kubernetes.io~projected/kube-api-access-hbsbr major:0 minor:241 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/98e7e1f3-2ca6-45de-a5d8-d27c22e257b1/volumes/kubernetes.io~secret/cert:{mountpoint:/var/lib/kubelet/pods/98e7e1f3-2ca6-45de-a5d8-d27c22e257b1/volumes/kubernetes.io~secret/cert major:0 minor:238 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/9962cf52-bd50-4a97-a699-4da03085eb81/volumes/kubernetes.io~projected/kube-api-access-qzmnh:{mountpoint:/var/lib/kubelet/pods/9962cf52-bd50-4a97-a699-4da03085eb81/volumes/kubernetes.io~projected/kube-api-access-qzmnh major:0 minor:60 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/9a20c40d-f5f4-4d96-85b6-be69a6cd1828/volumes/kubernetes.io~projected/kube-api-access-c6k8q:{mountpoint:/var/lib/kubelet/pods/9a20c40d-f5f4-4d96-85b6-be69a6cd1828/volumes/kubernetes.io~projected/kube-api-access-c6k8q major:0 minor:67 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/a3ceb095-c75c-43b3-9de5-7dda644eddb1/volumes/kubernetes.io~projected/kube-api-access-6rhvm:{mountpoint:/var/lib/kubelet/pods/a3ceb095-c75c-43b3-9de5-7dda644eddb1/volumes/kubernetes.io~projected/kube-api-access-6rhvm major:0 minor:65 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/a4d58318-42ba-4a9e-9c74-43fe90baa4a7/volumes/kubernetes.io~projected/kube-api-access-6sjmw:{mountpoint:/var/lib/kubelet/pods/a4d58318-42ba-4a9e-9c74-43fe90baa4a7/volumes/kubernetes.io~projected/kube-api-access-6sjmw major:0 minor:208 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/ac1dc607-3072-4dbc-87e9-db316bf42dff/volumes/kubernetes.io~projected/kube-api-access-9r47j:{mountpoint:/var/lib/kubelet/pods/ac1dc607-3072-4dbc-87e9-db316bf42dff/volumes/kubernetes.io~projected/kube-api-access-9r47j major:0 minor:66 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/b11d3d97-ad88-4aa3-aaee-e83e3cf12f6d/volumes/kubernetes.io~projected/kube-api-access-vtsfh:{mountpoint:/var/lib/kubelet/pods/b11d3d97-ad88-4aa3-aaee-e83e3cf12f6d/volumes/kubernetes.io~projected/kube-api-access-vtsfh major:0 minor:63 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/b86e8e45-3c2d-46b6-9ade-d390b56e4a75/volumes/kubernetes.io~projected/bound-sa-token:{mountpoint:/var/lib/kubelet/pods/b86e8e45-3c2d-46b6-9ade-d390b56e4a75/volumes/kubernetes.io~projected/bound-sa-token major:0 minor:213 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/b86e8e45-3c2d-46b6-9ade-d390b56e4a75/volumes/kubernetes.io~projected/kube-api-access-l8pqr:{mountpoint:/var/lib/kubelet/pods/b86e8e45-3c2d-46b6-9ade-d390b56e4a75/volumes/kubernetes.io~projected/kube-api-access-l8pqr major:0 minor:209 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/b86e8e45-3c2d-46b6-9ade-d390b56e4a75/volumes/kubernetes.io~secret/image-registry-private-configuration:{mountpoint:/var/lib/kubelet/pods/b86e8e45-3c2d-46b6-9ade-d390b56e4a75/volumes/kubernetes.io~secret/image-registry-private-configuration major:0 minor:204 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/b86e8e45-3c2d-46b6-9ade-d390b56e4a75/volumes/kubernetes.io~secret/installation-pull-secrets:{mountpoint:/var/lib/kubelet/pods/b86e8e45-3c2d-46b6-9ade-d390b56e4a75/volumes/kubernetes.io~secret/installation-pull-secrets major:0 minor:202 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/bd9e55af-64ad-4ffc-92c3-53b7bf654322/volumes/kubernetes.io~projected/kube-api-access-qt8c9:{mountpoint:/var/lib/kubelet/pods/bd9e55af-64ad-4ffc-92c3-53b7bf654322/volumes/kubernetes.io~projected/kube-api-access-qt8c9 major:0 minor:207 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/bd9e55af-64ad-4ffc-92c3-53b7bf654322/volumes/kubernetes.io~secret/serving-cert:{mountpoint:/var/lib/kubelet/pods/bd9e55af-64ad-4ffc-92c3-53b7bf654322/volumes/kubernetes.io~secret/serving-cert major:0 minor:206 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/d8930ee7-0ae0-48b5-8ce3-cb8820355cef/volumes/kubernetes.io~projected/kube-api-access-tltj6:{mountpoint:/var/lib/kubelet/pods/d8930ee7-0ae0-48b5-8ce3-cb8820355cef/volumes/kubernetes.io~projected/kube-api-access-tltj6 major:0 minor:231 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/d8930ee7-0ae0-48b5-8ce3-cb8820355cef/volumes/kubernetes.io~secret/serving-cert:{mountpoint:/var/lib/kubelet/pods/d8930ee7-0ae0-48b5-8ce3-cb8820355cef/volumes/kubernetes.io~secret/serving-cert major:0 minor:226 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/e1f629e6-6a96-444f-8329-e71bb98c9029/volumes/kubernetes.io~projected/kube-api-access-phkm5:{mountpoint:/var/lib/kubelet/pods/e1f629e6-6a96-444f-8329-e71bb98c9029/volumes/kubernetes.io~projected/kube-api-access-phkm5 major:0 minor:234 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/e1f629e6-6a96-444f-8329-e71bb98c9029/volumes/kubernetes.io~secret/default-certificate:{mountpoint:/var/lib/kubelet/pods/e1f629e6-6a96-444f-8329-e71bb98c9029/volumes/kubernetes.io~secret/default-certificate major:0 minor:225 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/e1f629e6-6a96-444f-8329-e71bb98c9029/volumes/kubernetes.io~secret/stats-auth:{mountpoint:/var/lib/kubelet/pods/e1f629e6-6a96-444f-8329-e71bb98c9029/volumes/kubernetes.io~secret/stats-auth major:0 minor:222 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/e30bf7f2-4c3a-4801-8ef1-f0611812a9d4/volumes/kubernetes.io~projected/kube-api-access-qszn6:{mountpoint:/var/lib/kubelet/pods/e30bf7f2-4c3a-4801-8ef1-f0611812a9d4/volumes/kubernetes.io~projected/kube-api-access-qszn6 major:0 minor:334 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/eca286e8-0b02-42af-9d15-1d8670b7362c/volumes/kubernetes.io~projected/kube-api-access-j2mgz:{mountpoint:/var/lib/kubelet/pods/eca286e8-0b02-42af-9d15-1d8670b7362c/volumes/kubernetes.io~projected/kube-api-access-j2mgz major:0 minor:61 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/f07abb3f-8900-4fa5-8d35-c7ab1d35582d/volumes/kubernetes.io~projected/kube-api-access-w2lch:{mountpoint:/var/lib/kubelet/pods/f07abb3f-8900-4fa5-8d35-c7ab1d35582d/volumes/kubernetes.io~projected/kube-api-access-w2lch major:0 minor:45 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/f660a256-a12d-4a4a-823d-4b0ff5dcb3da/volumes/kubernetes.io~secret/original-pull-secret:{mountpoint:/var/lib/kubelet/pods/f660a256-a12d-4a4a-823d-4b0ff5dcb3da/volumes/kubernetes.io~secret/original-pull-secret major:0 minor:293 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/fcf99170-f5cf-4987-97bd-1c30b1b37a35/volumes/kubernetes.io~projected/kube-api-access-qxmjj:{mountpoint:/var/lib/kubelet/pods/fcf99170-f5cf-4987-97bd-1c30b1b37a35/volumes/kubernetes.io~projected/kube-api-access-qxmjj major:0 minor:239 fsType:tmpfs blockSize:0} composefs_0-34:{mountpoint:/ major:0 minor:34 fsType:overlay blockSize:0} overlay_0-100:{mountpoint:/var/lib/containers/storage/overlay/f6d5f8c60e2a6983d7a4d9f1d2ecc320bcc577ee9659f77758c2bd50f2466178/merged major:0 minor:100 fsType:overlay blockSize:0} overlay_0-102:{mountpoint:/var/lib/containers/storage/overlay/38d49298bb065864444627468cc0408f495224e167904c9a45b0fac92122ad8e/merged major:0 minor:102 fsType:overlay blockSize:0} overlay_0-104:{mountpoint:/var/lib/containers/storage/overlay/3eda1b29992b3b892b215ead87d3227c4a6849ea93b0dba1682d5abb75a441be/merged major:0 minor:104 fsType:overlay blockSize:0} overlay_0-106:{mountpoint:/var/lib/containers/storage/overlay/36046fad1203c5afaf54de91af41b433a08e305f955052b9a344f5244d882ca0/merged major:0 minor:106 fsType:overlay blockSize:0} overlay_0-114:{mountpoint:/var/lib/containers/storage/overlay/9596444e8ba0c1bb9c6d8ad10d4f4eb8c6e28231aef060d0919b4794278a1273/merged major:0 minor:114 fsType:overlay blockSize:0} overlay_0-119:{mountpoint:/var/lib/containers/storage/overlay/7a29581f80eb1093107cc4d241be314f415904ec13fb1f37f13cee1fda5af295/merged major:0 minor:119 fsType:overlay blockSize:0} overlay_0-124:{mountpoint:/var/lib/containers/storage/overlay/466b4f566ce0bb8ed8e73709792bcf0776294f9f8ab1ac0a79fb34b52ba531b4/merged major:0 minor:124 fsType:overlay blockSize:0} overlay_0-127:{mountpoint:/var/lib/containers/storage/overlay/5c395a9943c5da992c7ec4c05a98b4e306d15f4160dfb85da9c1bb51c13d964f/merged major:0 minor:127 fsType:overlay blockSize:0} overlay_0-129:{mountpoint:/var/lib/containers/storage/overlay/3639e1ec51d637609eaf350f158cebeed8bdcf033931d961bfc6fd95738872e2/merged major:0 minor:129 fsType:overlay blockSize:0} overlay_0-137:{mountpoint:/var/lib/containers/storage/overlay/1edb294b25229d169d24ef55dbd743c694fd0361c5e243cf0a7e4379049d72a8/merged major:0 minor:137 fsType:overlay blockSize:0} overlay_0-141:{mountpoint:/var/lib/containers/storage/overlay/0bc27acec4e2adad6266b4030961d8fab1c95d316b811c8a05966beb278d8ff8/merged major:0 minor:141 fsType:overlay blockSize:0} overlay_0-145:{mountpoint:/var/lib/containers/storage/overlay/15d87eb3b65c4b3fc96975e317f1034ae69c005726c194a2087e04c30707deb7/merged major:0 minor:145 fsType:overlay blockSize:0} overlay_0-152:{mountpoint:/var/lib/containers/storage/overlay/442d9113d6837660e1d2e0c5947765e7a84efeb18805f46f87e0c9d81d3bbd54/merged major:0 minor:152 fsType:overlay blockSize:0} overlay_0-157:{mountpoint:/var/lib/containers/storage/overlay/4230dd475cd66b35de0e73c83ea99933c18be8bb0438333c5f6593931ae5d743/merged major:0 minor:157 fsType:overlay blockSize:0} overlay_0-162:{mountpoint:/var/lib/containers/storage/overlay/e87b11abe1542f227049cf796ccc854a324967c1a1a3b7d9c7bc3f7bf2547220/merged major:0 minor:162 fsType:overlay blockSize:0} overlay_0-167:{mountpoint:/var/lib/containers/storage/overlay/bbc3513a698b49ac28e02b0379fe687ec253f207a9f1c130220868d72237efc9/merged major:0 minor:167 fsType:overlay blockSize:0} overlay_0-172:{mountpoint:/var/lib/containers/storage/overlay/09c86bcc50329bc97de9653ab157591cf3ede761b2dad720db7fdd71bd7a1aa1/merged major:0 minor:172 fsType:overlay blockSize:0} overlay_0-177:{mountpoint:/var/lib/containers/storage/overlay/b5e67d32d17d8b9b14b00f400558fc13916667e138ad3947d436b9fef669e32c/merged major:0 minor:177 fsType:overlay blockSize:0} overlay_0-182:{mountpoint:/var/lib/containers/storage/overlay/e3ef27267ae144c9b73fb95c2d7112cecf7fc900e93ea93516efaf52db5b33bb/merged major:0 minor:182 fsType:overlay blockSize:0} overlay_0-187:{mountpoint:/var/lib/containers/storage/overlay/9a99e507110dd90cd4089aa69b2e46e9770f2121f69b2a94ec59361ed4c3c0ab/merged major:0 minor:187 fsType:overlay blockSize:0} overlay_0-192:{mountpoint:/var/lib/containers/storage/overlay/b3d6b8d3c61415d9247a7dfbd6a6057ea0990cb50957bcd78091467f6b56eff9/merged major:0 minor:192 fsType:overlay blockSize:0} overlay_0-248:{mountpoint:/var/lib/containers/storage/overlay/afb18008a025507112e8aae3517c18d6ebb72edd18a9ac1de3340acedd2741c0/merged major:0 minor:248 fsType:overlay blockSize:0} overlay_0-250:{mountpoint:/var/lib/containers/storage/overlay/315c117c7de19da276b93f1773f2df370f5b239cf2f3954086e304fb4cc5bb08/merged major:0 minor:250 fsType:overlay blockSize:0} overlay_0-252:{mountpoint:/var/lib/containers/storage/overlay/b348b7f766f07fb2ed78478fc69b601497467eed9b1b47704e4b078a47cf10dc/merged major:0 minor:252 fsType:overlay blockSize:0} overlay_0-254:{mountpoint:/var/lib/containers/storage/overlay/ba2ce19f79f0b33f3b88a2398e156530123bdce17edd77bb6fb0f56ea37c636b/merged major:0 minor:254 fsType:overlay blockSize:0} overlay_0-256:{mountpoint:/var/lib/containers/storage/overlay/87aef4b3471d0d44b60391a19290e44abddc70e5cb3e4f8a19120f9d938faa79/merged major:0 minor:256 fsType:overlay blockSize:0} overlay_0-258:{mountpoint:/var/lib/containers/storage/overlay/41be15090f4098f59eeb2e81d10ea6cd7793ea5ebabc6843f5ea2786453007dd/merged major:0 minor:258 fsType:overlay blockSize:0} overlay_0-260:{mountpoint:/var/lib/containers/storage/overlay/8fd87d92b18384a67399f1a0f21fbc59c04ed3a05e9d835a61efe7925d6cd732/merged major:0 minor:260 fsType:overlay blockSize:0} overlay_0-262:{mountpoint:/var/lib/containers/storage/overlay/126225f4169a159228f946e2f607b5af7b47e07a4acf57b6c22a6cb7775d2fff/merged major:0 minor:262 fsType:overlay blockSize:0} overlay_0-264:{mountpoint:/var/lib/containers/storage/overlay/d9b38c875c2021e7c56bfcf0b952c5956365b1def600c23cb0ca431f14432168/merged major:0 minor:264 fsType:overlay blockSize:0} overlay_0-266:{mountpoint:/var/lib/containers/storage/overlay/542bf663128e410ae89054a56d40e28b742f1eafb34e3268fac7a8677920c3a6/merged major:0 minor:266 fsType:overlay blockSize:0} overlay_0-268:{mountpoint:/var/lib/containers/storage/overlay/c14679b6f64132a680672a06204aee5804119489437631252f83a554b85f2f5e/merged major:0 minor:268 fsType:overlay blockSize:0} overlay_0-273:{mountpoint:/var/lib/containers/storage/overlay/3ddef62e3d54ec0196d0f8d946f8ac8b64a8803f5b58a5a75821a40d582e76f9/merged major:0 minor:273 fsType:overlay blockSize:0} overlay_0-284:{mountpoint:/var/lib/containers/storage/overlay/e789975a962be4c1993614350819f9abf1541c6137c35f8621d659d0139636f3/merged major:0 minor:284 fsType:overlay blockSize:0} overlay_0-286:{mountpoint:/var/lib/containers/storage/overlay/d85ba3096748c6fb856d972b74a4139c189a866accee7928cb946367ee78f7e8/merged major:0 minor:286 fsType:overlay blockSize:0} overlay_0-298:{mountpoint:/var/lib/containers/storage/overlay/b178f243feb0df309339ee996a594819479e3e3830d34a28d3bf9173ed8f6c6d/merged major:0 minor:298 fsType:overlay blockSize:0} overlay_0-300:{mountpoint:/var/lib/containers/storage/overlay/fb9d7224a2ddd5afdf5f12c14b8f12143f4995c05ca87b84d0f1192241d3196f/merged major:0 minor:300 fsType:overlay blockSize:0} overlay_0-302:{mountpoint:/var/lib/containers/storage/overlay/b3cfd13ab386e47fb8190b0e59f0c7ddc80ab98478b7398108ee73eb3b6a7cae/merged major:0 minor:302 fsType:overlay blockSize:0} overlay_0-304:{mountpoint:/var/lib/containers/storage/overlay/d8b4e4e0966e5a0eb292702b647a84cc374b91d2be278dabdd4c3ffefe54d1e7/merged major:0 minor:304 fsType:overlay blockSize:0} overlay_0-306:{mountpoint:/var/lib/containers/storage/overlay/691c08047895203c5f43a4dd5cd140c31148a34c9312fc24e93bf24dc2d6e550/merged major:0 minor:306 fsType:overlay blockSize:0} overlay_0-308:{mountpoint:/var/lib/containers/storage/overlay/1ae21e400b802b0e9980cd479b9a59f8ff406ae928e8fadfb18e6cababe1078d/merged major:0 minor:308 fsType:overlay blockSize:0} overlay_0-310:{mountpoint:/var/lib/containers/storage/overlay/6a85cb62a90e83484c363afaf81d5b82219dac4dc3ecb3b4c59851ba50d1ba45/merged major:0 minor:310 fsType:overlay blockSize:0} overlay_0-329:{mountpoint:/var/lib/containers/storage/overlay/f7aa5301b721ac39d0fd95881820b62a5de0928e10baaed1dbe0bde470db83a6/merged major:0 minor:329 fsType:overlay blockSize:0} overlay_0-342:{mountpoint:/var/lib/containers/storage/overlay/f74e1b5b876a0022991c1b37465fc0ea045a9706f25a506047a04ef3b3b84a4d/merged major:0 minor:342 fsType:overlay blockSize:0} overlay_0-344:{mountpoint:/var/lib/containers/storage/overlay/d64b4c431a44929ea6959cc511f17bb5fcdfd33c8569a20bb746b60952f47e0e/merged major:0 minor:344 fsType:overlay blockSize:0} overlay_0-346:{mountpoint:/var/lib/containers/storage/overlay/2f08fd9933e0115042ccb98ebb8c3ad6beb68b3bfda8c4e969739bcca5ac1ce3/merged major:0 minor:346 fsType:overlay blockSize:0} overlay_0-51:{mountpoint:/var/lib/containers/storage/overlay/cc931dde14aec0645c1b1c042432b0f5b35b248a0e47366df07859552538cebe/merged major:0 minor:51 fsType:overlay blockSize:0} overlay_0-53:{mountpoint:/var/lib/containers/storage/overlay/5ac1a0175c38bcff5b007ab1a85317523d30a39994a3ced1d891fdc79ec376d1/merged major:0 minor:53 fsType:overlay blockSize:0} overlay_0-70:{mountpoint:/var/lib/containers/storage/overlay/0c0c356d8d9fd442a981a213aaa1daae49854a4bbb96928841e945a128a9c942/merged major:0 minor:70 fsType:overlay blockSize:0} overlay_0-74:{mountpoint:/var/lib/containers/storage/overlay/4669f9f903627387828f63cb81b3a6fdad0876e8611efffc01f3f246dcfcae48/merged major:0 minor:74 fsType:overlay blockSize:0} overlay_0-81:{mountpoint:/var/lib/containers/storage/overlay/6957b0bd729f3e1caeb914f8adac025a62d94123f68b6bfb7507d522b0461280/merged major:0 minor:81 fsType:overlay blockSize:0} overlay_0-83:{mountpoint:/var/lib/containers/storage/overlay/3df43127f1f2b3118bfeef426e0faa5f7b7d8e6cc44a2f86132e23265f6ddbff/merged major:0 minor:83 fsType:overlay blockSize:0} overlay_0-87:{mountpoint:/var/lib/containers/storage/overlay/60c658334fb5983f6481dd7452755e30d35b1f15c8b9666fed50c47ff820cdda/merged major:0 minor:87 fsType:overlay blockSize:0} overlay_0-89:{mountpoint:/var/lib/containers/storage/overlay/9f912fc45c91c39465ba268ea9aadac23cd4209be527d2c6a4a1c343043b396b/merged major:0 minor:89 fsType:overlay blockSize:0} overlay_0-92:{mountpoint:/var/lib/containers/storage/overlay/8a0df52f060522482149473566a1680b00429641d891d27fd8d85f25c18d724f/merged major:0 minor:92 fsType:overlay blockSize:0} overlay_0-98:{mountpoint:/var/lib/containers/storage/overlay/21e97353a99638c2784c9520168c90189d1bae1870b1e4e8fdbb075f88d6687e/merged major:0 minor:98 fsType:overlay blockSize:0}]
Oct 30 12:57:04.289545 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.287579    4943 manager.go:217] Machine: {Timestamp:2026-10-30 12:57:04.287120336 +0000 UTC m=+0.075124056 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3101473 MemoryCapacity:33164337152 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec23c5d664d2c72415e815b707295ddc SystemUUID:ec23c5d6-64d2-c724-15e8-15b707295ddc BootID:a6bea421-76b6-4356-8a08-0ebde2fd07c8 Filesystems:[{Device:/run/containers/storage/overlay-containers/9387d634b449ee1cf65146c83f1fd3d16ba72de247544e9952f79af081174b7f/userdata/shm DeviceMajor:0 DeviceMinor:236 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/run/containers/storage/overlay-containers/afc6d32b630af29555dbee82da400929a106c898110d62a6e248dac42fa99dd1/userdata/shm DeviceMajor:0 DeviceMinor:47 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/51d01732-87a4-42b0-a6d5-a3fac689d7f6/volumes/kubernetes.io~secret/agent-certs DeviceMajor:0 DeviceMinor:58 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-145 DeviceMajor:0 DeviceMinor:145 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-157 DeviceMajor:0 DeviceMinor:157 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/4817ec63-f4d5-4420-9417-148346ff8bb5/volumes/kubernetes.io~projected/kube-api-access-7rwfx DeviceMajor:0 DeviceMinor:59 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/9a20c40d-f5f4-4d96-85b6-be69a6cd1828/volumes/kubernetes.io~projected/kube-api-access-c6k8q DeviceMajor:0 DeviceMinor:67 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/91591585f74679f5a203056eee04a9ff065c4244679fef41594168eba3ad7111/userdata/shm DeviceMajor:0 DeviceMinor:220 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/c501eda152de5cd4a5dedb5febbd60de386dfd9fd8cc677a2ebf5afebd205cad/userdata/shm DeviceMajor:0 DeviceMinor:223 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-304 DeviceMajor:0 DeviceMinor:304 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-104 DeviceMajor:0 DeviceMinor:104 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-119 DeviceMajor:0 DeviceMinor:119 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/b86e8e45-3c2d-46b6-9ade-d390b56e4a75/volumes/kubernetes.io~secret/image-registry-private-configuration DeviceMajor:0 DeviceMinor:204 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/27c55f84-4e9a-44de-94ea-02258d6bc747/volumes/kubernetes.io~projected/kube-api-access-hfnqv DeviceMajor:0 DeviceMinor:211 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/d8be95465e29616c512fecb557b5599168658616bffa156020d86ab0a30942f4/userdata/shm DeviceMajor:0 DeviceMinor:214 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/e1f629e6-6a96-444f-8329-e71bb98c9029/volumes/kubernetes.io~projected/kube-api-access-phkm5 DeviceMajor:0 DeviceMinor:234 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/f64856dcc52a80712efd8546500b0aa1dcf1f82708d8fc6b33eca1a2baa4320c/userdata/shm DeviceMajor:0 DeviceMinor:244 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-286 DeviceMajor:0 DeviceMinor:286 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/30f12e40f554fc638dc898f183ece14727f45446ed0a5f886f254f2034e43302/userdata/shm DeviceMajor:0 DeviceMinor:340 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-342 DeviceMajor:0 DeviceMinor:342 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-344 DeviceMajor:0 DeviceMinor:344 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/4ca8262c580200ac1c52c3d3783741b6914f7de30bbb7656c13e92eb6b55620e/userdata/shm DeviceMajor:0 DeviceMinor:72 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-152 DeviceMajor:0 DeviceMinor:152 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-254 DeviceMajor:0 DeviceMinor:254 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-258 DeviceMajor:0 DeviceMinor:258 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-187 DeviceMajor:0 DeviceMinor:187 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/1c405568-cb88-4485-ae2c-3614cc31ced6/volumes/kubernetes.io~secret/installation-pull-secrets DeviceMajor:0 DeviceMinor:228 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-308 DeviceMajor:0 DeviceMinor:308 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/56df6b59efaeea83af00f4daa02105b051a8b360001d76963a79837c1727270e/userdata/shm DeviceMajor:0 DeviceMinor:48 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/7950ed323f5ca09de76caa701be569bfe27144ce3c8f0be537a097a4d2ec2e47/userdata/shm DeviceMajor:0 DeviceMinor:76 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/921c5569e0b7dd8f4c9893322f95435bbff7ce00371964f243fd87193d22af85/userdata/shm DeviceMajor:0 DeviceMinor:95 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/23297e74-48f9-47b7-bb5a-80189bdbf2c4/volumes/kubernetes.io~projected/kube-api-access-jz42t DeviceMajor:0 DeviceMinor:229 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-114 DeviceMajor:0 DeviceMinor:114 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-137 DeviceMajor:0 DeviceMinor:137 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-74 DeviceMajor:0 DeviceMinor:74 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/d32290f4009892eecbff7fd69950ecfd46024944d1fb20ad7609255ba446c867/userdata/shm DeviceMajor:0 DeviceMinor:77 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/19b38c46-bfd4-4ed0-a764-f7b672f18cc6/volumes/kubernetes.io~projected/kube-api-access-qwmfv DeviceMajor:0 DeviceMinor:212 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/b86e8e45-3c2d-46b6-9ade-d390b56e4a75/volumes/kubernetes.io~projected/bound-sa-token DeviceMajor:0 DeviceMinor:213 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/15f0ea4b68ee48208edbca8d812819c1cfee317092c13305aa22c767e5ce6480/userdata/shm DeviceMajor:0 DeviceMinor:232 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:32 Capacity:16582168576 Type:vfs Inodes:1048576 HasInodes:true} {Device:/var/lib/kubelet/pods/9962cf52-bd50-4a97-a699-4da03085eb81/volumes/kubernetes.io~projected/kube-api-access-qzmnh DeviceMajor:0 DeviceMinor:60 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-248 DeviceMajor:0 DeviceMinor:248 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-250 DeviceMajor:0 DeviceMinor:250 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-268 DeviceMajor:0 DeviceMinor:268 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-260 DeviceMajor:0 DeviceMinor:260 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-264 DeviceMajor:0 DeviceMinor:264 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-284 DeviceMajor:0 DeviceMinor:284 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/56ff7ccd-573e-4bb2-a2b7-199a0346ed9f/volumes/kubernetes.io~empty-dir/tmp DeviceMajor:0 DeviceMinor:55 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/16402666-f516-49d1-be0d-f6ab7234e637/volumes/kubernetes.io~projected/kube-api-access-8xmnf DeviceMajor:0 DeviceMinor:62 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-192 DeviceMajor:0 DeviceMinor:192 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/59fa8cb30606c434c1df30aa0f941a6c6d80de50607501c039b61cfc0a8f5759/userdata/shm DeviceMajor:0 DeviceMinor:216 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-310 DeviceMajor:0 DeviceMinor:310 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/55fa4842-3eb4-4e52-a4fc-18876ca409d5/volumes/kubernetes.io~projected/kube-api-access-zg85t DeviceMajor:0 DeviceMinor:312 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-53 DeviceMajor:0 DeviceMinor:53 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/56ff7ccd-573e-4bb2-a2b7-199a0346ed9f/volumes/kubernetes.io~projected/kube-api-access-66dlr DeviceMajor:0 DeviceMinor:64 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/a3ceb095-c75c-43b3-9de5-7dda644eddb1/volumes/kubernetes.io~projected/kube-api-access-6rhvm DeviceMajor:0 DeviceMinor:65 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/bd9e55af-64ad-4ffc-92c3-53b7bf654322/volumes/kubernetes.io~secret/serving-cert DeviceMajor:0 DeviceMinor:206 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/8d767e44-14df-4ea6-b391-bdf265773535/volumes/kubernetes.io~projected/kube-api-access-9vr7r DeviceMajor:0 DeviceMinor:288 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/b86e8e45-3c2d-46b6-9ade-d390b56e4a75/volumes/kubernetes.io~secret/installation-pull-secrets DeviceMajor:0 DeviceMinor:202 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/eca286e8-0b02-42af-9d15-1d8670b7362c/volumes/kubernetes.io~projected/kube-api-access-j2mgz DeviceMajor:0 DeviceMinor:61 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/842ea7f7aa13c414c50b554ba5482c42c5614c8418f34fd986e33d90be9e34d0/userdata/shm DeviceMajor:0 DeviceMinor:68 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/f07abb3f-8900-4fa5-8d35-c7ab1d35582d/volumes/kubernetes.io~projected/kube-api-access-w2lch DeviceMajor:0 DeviceMinor:45 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/e1f629e6-6a96-444f-8329-e71bb98c9029/volumes/kubernetes.io~secret/stats-auth DeviceMajor:0 DeviceMinor:222 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-298 DeviceMajor:0 DeviceMinor:298 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:24 Capacity:16582168576 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/ac1dc607-3072-4dbc-87e9-db316bf42dff/volumes/kubernetes.io~projected/kube-api-access-9r47j DeviceMajor:0 DeviceMinor:66 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-167 DeviceMajor:0 DeviceMinor:167 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/301eebcb-3d48-4eef-a0de-a1173e23ad65/volumes/kubernetes.io~projected/kube-api-access-49z5c DeviceMajor:0 DeviceMinor:201 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/7ed65395-2cbd-46bf-81dc-fe307f3a9345/volumes/kubernetes.io~projected/kube-api-access-gz9d6 DeviceMajor:0 DeviceMinor:240 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-273 DeviceMajor:0 DeviceMinor:273 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:26 Capacity:6632869888 Type:vfs Inodes:819200 HasInodes:true} {Device:/run/containers/storage/overlay-containers/8a35b228406b2d0201049df742444e4ce6aceda6624f683a21b2f712acc51808/userdata/shm DeviceMajor:0 DeviceMinor:85 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-83 DeviceMajor:0 DeviceMinor:83 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-89 DeviceMajor:0 DeviceMinor:89 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-124 DeviceMajor:0 DeviceMinor:124 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-141 DeviceMajor:0 DeviceMinor:141 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/98e7e1f3-2ca6-45de-a5d8-d27c22e257b1/volumes/kubernetes.io~secret/cert DeviceMajor:0 DeviceMinor:238 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/e30bf7f2-4c3a-4801-8ef1-f0611812a9d4/volumes/kubernetes.io~projected/kube-api-access-qszn6 DeviceMajor:0 DeviceMinor:334 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:composefs_0-34 DeviceMajor:0 DeviceMinor:34 Capacity:6422528 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:overlay_0-51 DeviceMajor:0 DeviceMinor:51 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/4817ec63-f4d5-4420-9417-148346ff8bb5/volume-subpaths/run-systemd/ovnkube-controller/6 DeviceMajor:0 DeviceMinor:26 Capacity:6632869888 Type:vfs Inodes:819200 HasInodes:true} {Device:/run/containers/storage/overlay-containers/08239c831d96bfc6cf0bec4a44ccf8911e596dd61d29ac6eacc10da29b2e6b5e/userdata/shm DeviceMajor:0 DeviceMinor:218 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/d8930ee7-0ae0-48b5-8ce3-cb8820355cef/volumes/kubernetes.io~secret/serving-cert DeviceMajor:0 DeviceMinor:226 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-306 DeviceMajor:0 DeviceMinor:306 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/b11d3d97-ad88-4aa3-aaee-e83e3cf12f6d/volumes/kubernetes.io~projected/kube-api-access-vtsfh DeviceMajor:0 DeviceMinor:63 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-87 DeviceMajor:0 DeviceMinor:87 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-106 DeviceMajor:0 DeviceMinor:106 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/fcf99170-f5cf-4987-97bd-1c30b1b37a35/volumes/kubernetes.io~projected/kube-api-access-qxmjj DeviceMajor:0 DeviceMinor:239 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/78bd4381c30eb75c22521d05e237a70dfabe47221dacc0c74ebcbd5e36ff8fbf/userdata/shm DeviceMajor:0 DeviceMinor:296 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/e1f629e6-6a96-444f-8329-e71bb98c9029/volumes/kubernetes.io~secret/default-certificate DeviceMajor:0 DeviceMinor:225 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-266 DeviceMajor:0 DeviceMinor:266 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/72c23193bad92438cb473b27beb7f3f98a9a10064ec52310d78c6637bae06e28/userdata/shm DeviceMajor:0 DeviceMinor:79 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-81 DeviceMajor:0 DeviceMinor:81 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-172 DeviceMajor:0 DeviceMinor:172 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/bd9e55af-64ad-4ffc-92c3-53b7bf654322/volumes/kubernetes.io~projected/kube-api-access-qt8c9 DeviceMajor:0 DeviceMinor:207 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/98e7e1f3-2ca6-45de-a5d8-d27c22e257b1/volumes/kubernetes.io~projected/kube-api-access-hbsbr DeviceMajor:0 DeviceMinor:241 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/542a1406489aeed500a43e313ecb063b343fb53aa6b064ac09cbc4346db8987c/userdata/shm DeviceMajor:0 DeviceMinor:94 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-102 DeviceMajor:0 DeviceMinor:102 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-127 DeviceMajor:0 DeviceMinor:127 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-177 DeviceMajor:0 DeviceMinor:177 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/b86e8e45-3c2d-46b6-9ade-d390b56e4a75/volumes/kubernetes.io~projected/kube-api-access-l8pqr DeviceMajor:0 DeviceMinor:209 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/1c405568-cb88-4485-ae2c-3614cc31ced6/volumes/kubernetes.io~secret/image-registry-private-configuration DeviceMajor:0 DeviceMinor:227 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-162 DeviceMajor:0 DeviceMinor:162 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/4817ec63-f4d5-4420-9417-148346ff8bb5/volumes/kubernetes.io~secret/ovn-node-metrics-cert DeviceMajor:0 DeviceMinor:56 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/0905782c-26ad-4dc8-93cf-d6f211ea7af5/volumes/kubernetes.io~projected/kube-api-access-swxqr DeviceMajor:0 DeviceMinor:197 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-92 DeviceMajor:0 DeviceMinor:92 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-346 DeviceMajor:0 DeviceMinor:346 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/56ff7ccd-573e-4bb2-a2b7-199a0346ed9f/volumes/kubernetes.io~empty-dir/etc-tuned DeviceMajor:0 DeviceMinor:57 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/a4d58318-42ba-4a9e-9c74-43fe90baa4a7/volumes/kubernetes.io~projected/kube-api-access-6sjmw DeviceMajor:0 DeviceMinor:208 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/ef64188e81684686678c54e0cb2634c8321a9f8e65abacee44523832bc1b9c45/userdata/shm DeviceMajor:0 DeviceMinor:242 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/27c55f84-4e9a-44de-94ea-02258d6bc747/volumes/kubernetes.io~secret/serving-cert DeviceMajor:0 DeviceMinor:205 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/1c405568-cb88-4485-ae2c-3614cc31ced6/volumes/kubernetes.io~projected/bound-sa-token DeviceMajor:0 DeviceMinor:233 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/85f4c399-0089-4399-86e2-069834f89286/volumes/kubernetes.io~projected/kube-api-access-rls7j DeviceMajor:0 DeviceMinor:210 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-300 DeviceMajor:0 DeviceMinor:300 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-70 DeviceMajor:0 DeviceMinor:70 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-129 DeviceMajor:0 DeviceMinor:129 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-182 DeviceMajor:0 DeviceMinor:182 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-252 DeviceMajor:0 DeviceMinor:252 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/85f4c399-0089-4399-86e2-069834f89286/volumes/kubernetes.io~secret/serving-cert DeviceMajor:0 DeviceMinor:203 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/1c405568-cb88-4485-ae2c-3614cc31ced6/volumes/kubernetes.io~projected/kube-api-access-54kft DeviceMajor:0 DeviceMinor:230 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/d8930ee7-0ae0-48b5-8ce3-cb8820355cef/volumes/kubernetes.io~projected/kube-api-access-tltj6 DeviceMajor:0 DeviceMinor:231 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/e463ce667e1fda351b2fee3fbb5b5a5194516d6c0afc229100c0f8501e19fe57/userdata/shm DeviceMajor:0 DeviceMinor:246 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-262 DeviceMajor:0 DeviceMinor:262 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/f660a256-a12d-4a4a-823d-4b0ff5dcb3da/volumes/kubernetes.io~secret/original-pull-secret DeviceMajor:0 DeviceMinor:293 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/3389cdf237dd6b19ba280bf5738bd8a7a90c3f3d03e56b7ea402b586ead69d51/userdata/shm DeviceMajor:0 DeviceMinor:294 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-329 DeviceMajor:0 DeviceMinor:329 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-256 DeviceMajor:0 DeviceMinor:256 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-98 DeviceMajor:0 DeviceMinor:98 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-100 DeviceMajor:0 DeviceMinor:100 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-302 DeviceMajor:0 DeviceMinor:302 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:08239c831d96bfc MacAddress:ca:8b:62:60:76:b2 Speed:10000 Mtu:8901} {Name:15f0ea4b68ee482 MacAddress:0a:15:ae:57:a8:67 Speed:10000 Mtu:8901} {Name:30f12e40f554fc6 MacAddress:8e:1c:2b:b7:05:74 Speed:10000 Mtu:8901} {Name:3389cdf237dd6b1 MacAddress:3a:28:99:f0:2a:20 Speed:10000 Mtu:8901} {Name:59fa8cb30606c43 MacAddress:b2:10:46:4e:e9:0d Speed:10000 Mtu:8901} {Name:78bd4381c30eb75 MacAddress:c2:71:43:e7:df:28 Speed:10000 Mtu:8901} {Name:91591585f74679f MacAddress:62:0a:36:32:4e:a0 Speed:10000 Mtu:8901} {Name:9387d634b449ee1 MacAddress:ee:b0:4f:8e:fd:2b Speed:10000 Mtu:8901} {Name:br-ex MacAddress:02:ff:f5:4a:d8:11 Speed:0 Mtu:9001} {Name:br-int MacAddress:f2:ea:25:8e:8e:fa Speed:0 Mtu:8901} {Name:c501eda152de5cd MacAddress:ba:56:17:94:c1:a8 Speed:10000 Mtu:8901} {Name:d8be95465e29616 MacAddress:6e:05:79:6d:1e:cb Speed:10000 Mtu:8901} {Name:e463ce667e1fda3 MacAddress:12:11:bc:97:2c:64 Speed:10000 Mtu:8901} {Name:ef64188e8168468 MacAddress:76:c2:77:96:27:d5 Speed:10000 Mtu:8901} {Name:ens5 MacAddress:02:ff:f5:4a:d8:11 Speed:0 Mtu:9001} {Name:f64856dcc52a807 MacAddress:ae:62:21:6c:02:48 Speed:10000 Mtu:8901} {Name:genev_sys_6081 MacAddress:4a:d3:e1:aa:44:59 Speed:0 Mtu:65000} {Name:ovn-k8s-mp0 MacAddress:0a:58:0a:84:00:02 Speed:0 Mtu:8901} {Name:ovs-system MacAddress:46:07:dc:bb:70:9f Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164337152 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Oct 30 12:57:04.289545 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.287948    4943 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Oct 30 12:57:04.289545 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.287991    4943 manager.go:233] Version: {KernelVersion:5.14.0-687.60.1.el9_8.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.8.20261023-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Oct 30 12:57:04.289545 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.288251    4943 container_manager_linux.go:275] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Oct 30 12:57:04.289545 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.288282    4943 container_manager_linux.go:280] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-142-84.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"/system.slice","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{},"system-reserved-compressible":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Oct 30 12:57:04.289545 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.288407    4943 topology_manager.go:143] "Creating topology manager with none policy"
Oct 30 12:57:04.289545 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.288415    4943 container_manager_linux.go:311] "Creating device plugin manager"
Oct 30 12:57:04.289545 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.288424    4943 manager.go:144] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Oct 30 12:57:04.289545 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.288437    4943 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Oct 30 12:57:04.289545 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.288456    4943 container_manager_linux.go:320] "Creating Dynamic Resource Allocation (DRA) manager"
Oct 30 12:57:04.289545 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.288590    4943 state_mem.go:41] "Initialized" logger="CPUManager state memory"
Oct 30 12:57:04.289545 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.288679    4943 server.go:1235] "Using root directory" path="/var/lib/kubelet"
Oct 30 12:57:04.289545 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.288709    4943 kubelet.go:493] "Attempting to sync node with API server"
Oct 30 12:57:04.289545 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.288719    4943 kubelet.go:394] "Adding static pod path" path="/etc/kubernetes/manifests"
Oct 30 12:57:04.289545 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.288736    4943 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Oct 30 12:57:04.289545 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.288744    4943 kubelet.go:405] "Adding apiserver pod source"
Oct 30 12:57:04.289545 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.288752    4943 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Oct 30 12:57:04.289545 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.289369    4943 state_checkpoint.go:81] "State checkpoint: restored pod resource state from checkpoint"
Oct 30 12:57:04.289545 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.289379    4943 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Oct 30 12:57:04.290506 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.289836    4943 kuberuntime_manager.go:304] "Container runtime initialized" containerRuntime="cri-o" version="1.35.10-3.rhaos4.22.git9383502.el9" apiVersion="v1"
Oct 30 12:57:04.292194 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.292181    4943 state_checkpoint.go:81] "State checkpoint: restored pod resource state from checkpoint"
Oct 30 12:57:04.292232 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.292195    4943 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Oct 30 12:57:04.292312 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.292302    4943 certificate_store.go:147] "Loading cert/key pair from a file" filePath="/var/lib/kubelet/pki/kubelet-server-current.pem"
Oct 30 12:57:04.292489 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.292479    4943 kubelet.go:963] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Oct 30 12:57:04.292521 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.292507    4943 kubelet.go:990] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Oct 30 12:57:04.292613 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.292604    4943 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Oct 30 12:57:04.292667 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.292622    4943 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Oct 30 12:57:04.292667 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.292630    4943 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Oct 30 12:57:04.292667 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.292638    4943 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Oct 30 12:57:04.292667 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.292647    4943 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Oct 30 12:57:04.292667 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.292656    4943 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Oct 30 12:57:04.292667 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.292665    4943 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Oct 30 12:57:04.292901 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.292673    4943 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Oct 30 12:57:04.292901 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.292680    4943 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Oct 30 12:57:04.292901 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.292686    4943 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Oct 30 12:57:04.292901 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.292699    4943 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Oct 30 12:57:04.292901 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.292709    4943 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Oct 30 12:57:04.292901 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.292743    4943 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Oct 30 12:57:04.292901 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.292750    4943 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Oct 30 12:57:04.296190 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.296178    4943 server.go:1263] "Started kubelet"
Oct 30 12:57:04.297221 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.296242    4943 server.go:182] "Starting to listen" address="0.0.0.0" port=10250
Oct 30 12:57:04.296791 ip-10-0-142-84 systemd[1]: Started Kubernetes Kubelet.
Oct 30 12:57:04.297555 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.296327    4943 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Oct 30 12:57:04.297641 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.297578    4943 server_v1.go:49] "podresources" method="list" useActivePods=true
Oct 30 12:57:04.297906 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.297882    4943 server.go:254] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Oct 30 12:57:04.298615 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.298601    4943 server.go:317] "Adding debug handlers to kubelet server"
Oct 30 12:57:04.302046 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.302026    4943 fs_resource_analyzer.go:69] "Starting FS ResourceAnalyzer"
Oct 30 12:57:04.302254 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.302232    4943 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Oct 30 12:57:04.302361 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.302314    4943 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-10-29 12:51:25 +0000 UTC" deadline="2028-03-26 01:18:41.230389382 +0000 UTC"
Oct 30 12:57:04.302361 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.302349    4943 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12300h21m36.928043494s"
Oct 30 12:57:04.302500 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.302482    4943 volume_manager.go:309] "The desired_state_of_world populator starts"
Oct 30 12:57:04.302560 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.302501    4943 volume_manager.go:311] "Starting Kubelet Volume Manager"
Oct 30 12:57:04.302612 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.302575    4943 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Oct 30 12:57:04.302669 ip-10-0-142-84 kubenswrapper[4943]: E1030 12:57:04.302640    4943 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-142-84.ec2.internal\" not found"
Oct 30 12:57:04.303359 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.303285    4943 factory.go:55] Registering systemd factory
Oct 30 12:57:04.303359 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.303308    4943 factory.go:223] Registration of the systemd container factory successfully
Oct 30 12:57:04.303598 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.303582    4943 factory.go:153] Registering CRI-O factory
Oct 30 12:57:04.303668 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.303602    4943 factory.go:223] Registration of the crio container factory successfully
Oct 30 12:57:04.303668 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.303655    4943 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Oct 30 12:57:04.303793 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.303672    4943 factory.go:103] Registering Raw factory
Oct 30 12:57:04.303793 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.303690    4943 manager.go:1199] Started watching for new ooms in manager
Oct 30 12:57:04.304067 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.304053    4943 manager.go:319] Starting recovery of all containers
Oct 30 12:57:04.306907 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.305537    4943 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="27c55f84-4e9a-44de-94ea-02258d6bc747" volumeName="kubernetes.io/secret/27c55f84-4e9a-44de-94ea-02258d6bc747-serving-cert" seLinuxMountContext=""
Oct 30 12:57:04.306907 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.305582    4943 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4817ec63-f4d5-4420-9417-148346ff8bb5" volumeName="kubernetes.io/secret/4817ec63-f4d5-4420-9417-148346ff8bb5-ovn-node-metrics-cert" seLinuxMountContext=""
Oct 30 12:57:04.306907 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.305599    4943 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="bd9e55af-64ad-4ffc-92c3-53b7bf654322" volumeName="kubernetes.io/configmap/bd9e55af-64ad-4ffc-92c3-53b7bf654322-trusted-ca-bundle" seLinuxMountContext=""
Oct 30 12:57:04.306907 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.305613    4943 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="bd9e55af-64ad-4ffc-92c3-53b7bf654322" volumeName="kubernetes.io/secret/bd9e55af-64ad-4ffc-92c3-53b7bf654322-serving-cert" seLinuxMountContext=""
Oct 30 12:57:04.306907 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.305630    4943 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d8930ee7-0ae0-48b5-8ce3-cb8820355cef" volumeName="kubernetes.io/configmap/d8930ee7-0ae0-48b5-8ce3-cb8820355cef-config" seLinuxMountContext=""
Oct 30 12:57:04.306907 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.305644    4943 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="19b38c46-bfd4-4ed0-a764-f7b672f18cc6" volumeName="kubernetes.io/projected/19b38c46-bfd4-4ed0-a764-f7b672f18cc6-kube-api-access-qwmfv" seLinuxMountContext=""
Oct 30 12:57:04.306907 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.305661    4943 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1c405568-cb88-4485-ae2c-3614cc31ced6" volumeName="kubernetes.io/configmap/1c405568-cb88-4485-ae2c-3614cc31ced6-trusted-ca" seLinuxMountContext=""
Oct 30 12:57:04.306907 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.305675    4943 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9a20c40d-f5f4-4d96-85b6-be69a6cd1828" volumeName="kubernetes.io/configmap/9a20c40d-f5f4-4d96-85b6-be69a6cd1828-multus-daemon-config" seLinuxMountContext=""
Oct 30 12:57:04.306907 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.305690    4943 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b86e8e45-3c2d-46b6-9ade-d390b56e4a75" volumeName="kubernetes.io/configmap/b86e8e45-3c2d-46b6-9ade-d390b56e4a75-trusted-ca" seLinuxMountContext=""
Oct 30 12:57:04.306907 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.305703    4943 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="bd9e55af-64ad-4ffc-92c3-53b7bf654322" volumeName="kubernetes.io/empty-dir/bd9e55af-64ad-4ffc-92c3-53b7bf654322-snapshots" seLinuxMountContext=""
Oct 30 12:57:04.306907 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.305716    4943 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b86e8e45-3c2d-46b6-9ade-d390b56e4a75" volumeName="kubernetes.io/empty-dir/b86e8e45-3c2d-46b6-9ade-d390b56e4a75-ca-trust-extracted" seLinuxMountContext=""
Oct 30 12:57:04.306907 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.305731    4943 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1c405568-cb88-4485-ae2c-3614cc31ced6" volumeName="kubernetes.io/empty-dir/1c405568-cb88-4485-ae2c-3614cc31ced6-ca-trust-extracted" seLinuxMountContext=""
Oct 30 12:57:04.306907 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.305744    4943 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1c405568-cb88-4485-ae2c-3614cc31ced6" volumeName="kubernetes.io/projected/1c405568-cb88-4485-ae2c-3614cc31ced6-kube-api-access-54kft" seLinuxMountContext=""
Oct 30 12:57:04.306907 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.305756    4943 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4817ec63-f4d5-4420-9417-148346ff8bb5" volumeName="kubernetes.io/configmap/4817ec63-f4d5-4420-9417-148346ff8bb5-ovnkube-config" seLinuxMountContext=""
Oct 30 12:57:04.306907 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.305783    4943 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="85f4c399-0089-4399-86e2-069834f89286" volumeName="kubernetes.io/projected/85f4c399-0089-4399-86e2-069834f89286-kube-api-access-rls7j" seLinuxMountContext=""
Oct 30 12:57:04.306907 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.305798    4943 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9962cf52-bd50-4a97-a699-4da03085eb81" volumeName="kubernetes.io/configmap/9962cf52-bd50-4a97-a699-4da03085eb81-whereabouts-flatfile-configmap" seLinuxMountContext=""
Oct 30 12:57:04.306907 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.305811    4943 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ac1dc607-3072-4dbc-87e9-db316bf42dff" volumeName="kubernetes.io/configmap/ac1dc607-3072-4dbc-87e9-db316bf42dff-iptables-alerter-script" seLinuxMountContext=""
Oct 30 12:57:04.306907 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.305824    4943 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d8930ee7-0ae0-48b5-8ce3-cb8820355cef" volumeName="kubernetes.io/configmap/d8930ee7-0ae0-48b5-8ce3-cb8820355cef-trusted-ca" seLinuxMountContext=""
Oct 30 12:57:04.306907 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.305836    4943 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="85f4c399-0089-4399-86e2-069834f89286" volumeName="kubernetes.io/secret/85f4c399-0089-4399-86e2-069834f89286-serving-cert" seLinuxMountContext=""
Oct 30 12:57:04.306907 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.305849    4943 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="8d767e44-14df-4ea6-b391-bdf265773535" volumeName="kubernetes.io/projected/8d767e44-14df-4ea6-b391-bdf265773535-kube-api-access-9vr7r" seLinuxMountContext=""
Oct 30 12:57:04.306907 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.305861    4943 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9a20c40d-f5f4-4d96-85b6-be69a6cd1828" volumeName="kubernetes.io/projected/9a20c40d-f5f4-4d96-85b6-be69a6cd1828-kube-api-access-c6k8q" seLinuxMountContext=""
Oct 30 12:57:04.306907 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.305873    4943 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b86e8e45-3c2d-46b6-9ade-d390b56e4a75" volumeName="kubernetes.io/configmap/b86e8e45-3c2d-46b6-9ade-d390b56e4a75-registry-certificates" seLinuxMountContext=""
Oct 30 12:57:04.306907 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.305887    4943 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b86e8e45-3c2d-46b6-9ade-d390b56e4a75" volumeName="kubernetes.io/projected/b86e8e45-3c2d-46b6-9ade-d390b56e4a75-bound-sa-token" seLinuxMountContext=""
Oct 30 12:57:04.306907 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.305901    4943 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b86e8e45-3c2d-46b6-9ade-d390b56e4a75" volumeName="kubernetes.io/secret/b86e8e45-3c2d-46b6-9ade-d390b56e4a75-installation-pull-secrets" seLinuxMountContext=""
Oct 30 12:57:04.306907 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.305913    4943 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="bd9e55af-64ad-4ffc-92c3-53b7bf654322" volumeName="kubernetes.io/configmap/bd9e55af-64ad-4ffc-92c3-53b7bf654322-service-ca-bundle" seLinuxMountContext=""
Oct 30 12:57:04.306907 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.305925    4943 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f07abb3f-8900-4fa5-8d35-c7ab1d35582d" volumeName="kubernetes.io/projected/f07abb3f-8900-4fa5-8d35-c7ab1d35582d-kube-api-access-w2lch" seLinuxMountContext=""
Oct 30 12:57:04.306907 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.305938    4943 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="27c55f84-4e9a-44de-94ea-02258d6bc747" volumeName="kubernetes.io/configmap/27c55f84-4e9a-44de-94ea-02258d6bc747-config" seLinuxMountContext=""
Oct 30 12:57:04.306907 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.305950    4943 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9962cf52-bd50-4a97-a699-4da03085eb81" volumeName="kubernetes.io/projected/9962cf52-bd50-4a97-a699-4da03085eb81-kube-api-access-qzmnh" seLinuxMountContext=""
Oct 30 12:57:04.306907 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.305969    4943 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ac1dc607-3072-4dbc-87e9-db316bf42dff" volumeName="kubernetes.io/projected/ac1dc607-3072-4dbc-87e9-db316bf42dff-kube-api-access-9r47j" seLinuxMountContext=""
Oct 30 12:57:04.306907 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.305983    4943 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e1f629e6-6a96-444f-8329-e71bb98c9029" volumeName="kubernetes.io/secret/e1f629e6-6a96-444f-8329-e71bb98c9029-stats-auth" seLinuxMountContext=""
Oct 30 12:57:04.306907 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.306015    4943 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="fcf99170-f5cf-4987-97bd-1c30b1b37a35" volumeName="kubernetes.io/projected/fcf99170-f5cf-4987-97bd-1c30b1b37a35-kube-api-access-qxmjj" seLinuxMountContext=""
Oct 30 12:57:04.306907 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.306027    4943 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0905782c-26ad-4dc8-93cf-d6f211ea7af5" volumeName="kubernetes.io/projected/0905782c-26ad-4dc8-93cf-d6f211ea7af5-kube-api-access-swxqr" seLinuxMountContext=""
Oct 30 12:57:04.306907 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.306040    4943 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="85f4c399-0089-4399-86e2-069834f89286" volumeName="kubernetes.io/configmap/85f4c399-0089-4399-86e2-069834f89286-config" seLinuxMountContext=""
Oct 30 12:57:04.306907 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.306053    4943 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="fcf99170-f5cf-4987-97bd-1c30b1b37a35" volumeName="kubernetes.io/configmap/fcf99170-f5cf-4987-97bd-1c30b1b37a35-config-volume" seLinuxMountContext=""
Oct 30 12:57:04.306907 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.306066    4943 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="fcf99170-f5cf-4987-97bd-1c30b1b37a35" volumeName="kubernetes.io/empty-dir/fcf99170-f5cf-4987-97bd-1c30b1b37a35-tmp-dir" seLinuxMountContext=""
Oct 30 12:57:04.306907 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.306079    4943 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="23297e74-48f9-47b7-bb5a-80189bdbf2c4" volumeName="kubernetes.io/projected/23297e74-48f9-47b7-bb5a-80189bdbf2c4-kube-api-access-jz42t" seLinuxMountContext=""
Oct 30 12:57:04.306907 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.306091    4943 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="51d01732-87a4-42b0-a6d5-a3fac689d7f6" volumeName="kubernetes.io/secret/51d01732-87a4-42b0-a6d5-a3fac689d7f6-agent-certs" seLinuxMountContext=""
Oct 30 12:57:04.306907 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.306104    4943 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="56ff7ccd-573e-4bb2-a2b7-199a0346ed9f" volumeName="kubernetes.io/empty-dir/56ff7ccd-573e-4bb2-a2b7-199a0346ed9f-tmp" seLinuxMountContext=""
Oct 30 12:57:04.306907 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.306116    4943 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b86e8e45-3c2d-46b6-9ade-d390b56e4a75" volumeName="kubernetes.io/secret/b86e8e45-3c2d-46b6-9ade-d390b56e4a75-image-registry-private-configuration" seLinuxMountContext=""
Oct 30 12:57:04.306907 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.306129    4943 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="98e7e1f3-2ca6-45de-a5d8-d27c22e257b1" volumeName="kubernetes.io/projected/98e7e1f3-2ca6-45de-a5d8-d27c22e257b1-kube-api-access-hbsbr" seLinuxMountContext=""
Oct 30 12:57:04.306907 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.306143    4943 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4817ec63-f4d5-4420-9417-148346ff8bb5" volumeName="kubernetes.io/projected/4817ec63-f4d5-4420-9417-148346ff8bb5-kube-api-access-7rwfx" seLinuxMountContext=""
Oct 30 12:57:04.306907 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.306155    4943 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="55fa4842-3eb4-4e52-a4fc-18876ca409d5" volumeName="kubernetes.io/projected/55fa4842-3eb4-4e52-a4fc-18876ca409d5-kube-api-access-zg85t" seLinuxMountContext=""
Oct 30 12:57:04.306907 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.306177    4943 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d8930ee7-0ae0-48b5-8ce3-cb8820355cef" volumeName="kubernetes.io/projected/d8930ee7-0ae0-48b5-8ce3-cb8820355cef-kube-api-access-tltj6" seLinuxMountContext=""
Oct 30 12:57:04.306907 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.306189    4943 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d8930ee7-0ae0-48b5-8ce3-cb8820355cef" volumeName="kubernetes.io/secret/d8930ee7-0ae0-48b5-8ce3-cb8820355cef-serving-cert" seLinuxMountContext=""
Oct 30 12:57:04.306907 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.306201    4943 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e1f629e6-6a96-444f-8329-e71bb98c9029" volumeName="kubernetes.io/projected/e1f629e6-6a96-444f-8329-e71bb98c9029-kube-api-access-phkm5" seLinuxMountContext=""
Oct 30 12:57:04.306907 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.306211    4943 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="eca286e8-0b02-42af-9d15-1d8670b7362c" volumeName="kubernetes.io/projected/eca286e8-0b02-42af-9d15-1d8670b7362c-kube-api-access-j2mgz" seLinuxMountContext=""
Oct 30 12:57:04.306907 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.306225    4943 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="19b38c46-bfd4-4ed0-a764-f7b672f18cc6" volumeName="kubernetes.io/configmap/19b38c46-bfd4-4ed0-a764-f7b672f18cc6-telemetry-config" seLinuxMountContext=""
Oct 30 12:57:04.306907 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.306238    4943 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1c405568-cb88-4485-ae2c-3614cc31ced6" volumeName="kubernetes.io/configmap/1c405568-cb88-4485-ae2c-3614cc31ced6-registry-certificates" seLinuxMountContext=""
Oct 30 12:57:04.306907 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.306250    4943 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b86e8e45-3c2d-46b6-9ade-d390b56e4a75" volumeName="kubernetes.io/projected/b86e8e45-3c2d-46b6-9ade-d390b56e4a75-kube-api-access-l8pqr" seLinuxMountContext=""
Oct 30 12:57:04.306907 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.306288    4943 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1c405568-cb88-4485-ae2c-3614cc31ced6" volumeName="kubernetes.io/secret/1c405568-cb88-4485-ae2c-3614cc31ced6-image-registry-private-configuration" seLinuxMountContext=""
Oct 30 12:57:04.306907 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.306300    4943 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="98e7e1f3-2ca6-45de-a5d8-d27c22e257b1" volumeName="kubernetes.io/secret/98e7e1f3-2ca6-45de-a5d8-d27c22e257b1-cert" seLinuxMountContext=""
Oct 30 12:57:04.306907 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.306313    4943 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9a20c40d-f5f4-4d96-85b6-be69a6cd1828" volumeName="kubernetes.io/configmap/9a20c40d-f5f4-4d96-85b6-be69a6cd1828-cni-binary-copy" seLinuxMountContext=""
Oct 30 12:57:04.306907 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.306326    4943 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="301eebcb-3d48-4eef-a0de-a1173e23ad65" volumeName="kubernetes.io/projected/301eebcb-3d48-4eef-a0de-a1173e23ad65-kube-api-access-49z5c" seLinuxMountContext=""
Oct 30 12:57:04.306907 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.306340    4943 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4883c26d-0877-4870-b0ee-406f5bfa8cf4" volumeName="kubernetes.io/configmap/4883c26d-0877-4870-b0ee-406f5bfa8cf4-nginx-conf" seLinuxMountContext=""
Oct 30 12:57:04.306907 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.306353    4943 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="56ff7ccd-573e-4bb2-a2b7-199a0346ed9f" volumeName="kubernetes.io/projected/56ff7ccd-573e-4bb2-a2b7-199a0346ed9f-kube-api-access-66dlr" seLinuxMountContext=""
Oct 30 12:57:04.306907 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.306365    4943 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9962cf52-bd50-4a97-a699-4da03085eb81" volumeName="kubernetes.io/configmap/9962cf52-bd50-4a97-a699-4da03085eb81-cni-sysctl-allowlist" seLinuxMountContext=""
Oct 30 12:57:04.306907 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.306407    4943 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e1f629e6-6a96-444f-8329-e71bb98c9029" volumeName="kubernetes.io/secret/e1f629e6-6a96-444f-8329-e71bb98c9029-default-certificate" seLinuxMountContext=""
Oct 30 12:57:04.306907 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.306420    4943 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="eca286e8-0b02-42af-9d15-1d8670b7362c" volumeName="kubernetes.io/empty-dir/eca286e8-0b02-42af-9d15-1d8670b7362c-tmp-dir" seLinuxMountContext=""
Oct 30 12:57:04.306907 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.306434    4943 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="16402666-f516-49d1-be0d-f6ab7234e637" volumeName="kubernetes.io/configmap/16402666-f516-49d1-be0d-f6ab7234e637-serviceca" seLinuxMountContext=""
Oct 30 12:57:04.306907 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.306448    4943 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="16402666-f516-49d1-be0d-f6ab7234e637" volumeName="kubernetes.io/projected/16402666-f516-49d1-be0d-f6ab7234e637-kube-api-access-8xmnf" seLinuxMountContext=""
Oct 30 12:57:04.306907 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.306461    4943 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="27c55f84-4e9a-44de-94ea-02258d6bc747" volumeName="kubernetes.io/projected/27c55f84-4e9a-44de-94ea-02258d6bc747-kube-api-access-hfnqv" seLinuxMountContext=""
Oct 30 12:57:04.306907 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.306474    4943 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4817ec63-f4d5-4420-9417-148346ff8bb5" volumeName="kubernetes.io/configmap/4817ec63-f4d5-4420-9417-148346ff8bb5-env-overrides" seLinuxMountContext=""
Oct 30 12:57:04.306907 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.306490    4943 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7ed65395-2cbd-46bf-81dc-fe307f3a9345" volumeName="kubernetes.io/projected/7ed65395-2cbd-46bf-81dc-fe307f3a9345-kube-api-access-gz9d6" seLinuxMountContext=""
Oct 30 12:57:04.306907 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.306502    4943 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9962cf52-bd50-4a97-a699-4da03085eb81" volumeName="kubernetes.io/configmap/9962cf52-bd50-4a97-a699-4da03085eb81-cni-binary-copy" seLinuxMountContext=""
Oct 30 12:57:04.306907 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.306516    4943 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a3ceb095-c75c-43b3-9de5-7dda644eddb1" volumeName="kubernetes.io/projected/a3ceb095-c75c-43b3-9de5-7dda644eddb1-kube-api-access-6rhvm" seLinuxMountContext=""
Oct 30 12:57:04.306907 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.306529    4943 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b11d3d97-ad88-4aa3-aaee-e83e3cf12f6d" volumeName="kubernetes.io/projected/b11d3d97-ad88-4aa3-aaee-e83e3cf12f6d-kube-api-access-vtsfh" seLinuxMountContext=""
Oct 30 12:57:04.306907 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.306542    4943 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="bd9e55af-64ad-4ffc-92c3-53b7bf654322" volumeName="kubernetes.io/empty-dir/bd9e55af-64ad-4ffc-92c3-53b7bf654322-tmp" seLinuxMountContext=""
Oct 30 12:57:04.306907 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.306556    4943 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="bd9e55af-64ad-4ffc-92c3-53b7bf654322" volumeName="kubernetes.io/projected/bd9e55af-64ad-4ffc-92c3-53b7bf654322-kube-api-access-qt8c9" seLinuxMountContext=""
Oct 30 12:57:04.306907 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.306570    4943 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="51d01732-87a4-42b0-a6d5-a3fac689d7f6" volumeName="kubernetes.io/configmap/51d01732-87a4-42b0-a6d5-a3fac689d7f6-konnectivity-ca" seLinuxMountContext=""
Oct 30 12:57:04.306907 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.306583    4943 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="55fa4842-3eb4-4e52-a4fc-18876ca409d5" volumeName="kubernetes.io/empty-dir/55fa4842-3eb4-4e52-a4fc-18876ca409d5-data-volume" seLinuxMountContext=""
Oct 30 12:57:04.306907 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.306595    4943 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="56ff7ccd-573e-4bb2-a2b7-199a0346ed9f" volumeName="kubernetes.io/empty-dir/56ff7ccd-573e-4bb2-a2b7-199a0346ed9f-etc-tuned" seLinuxMountContext=""
Oct 30 12:57:04.306907 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.306603    4943 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e30bf7f2-4c3a-4801-8ef1-f0611812a9d4" volumeName="kubernetes.io/projected/e30bf7f2-4c3a-4801-8ef1-f0611812a9d4-kube-api-access-qszn6" seLinuxMountContext=""
Oct 30 12:57:04.306907 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.306611    4943 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f660a256-a12d-4a4a-823d-4b0ff5dcb3da" volumeName="kubernetes.io/secret/f660a256-a12d-4a4a-823d-4b0ff5dcb3da-original-pull-secret" seLinuxMountContext=""
Oct 30 12:57:04.306907 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.306618    4943 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1c405568-cb88-4485-ae2c-3614cc31ced6" volumeName="kubernetes.io/projected/1c405568-cb88-4485-ae2c-3614cc31ced6-bound-sa-token" seLinuxMountContext=""
Oct 30 12:57:04.306907 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.306625    4943 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1c405568-cb88-4485-ae2c-3614cc31ced6" volumeName="kubernetes.io/secret/1c405568-cb88-4485-ae2c-3614cc31ced6-installation-pull-secrets" seLinuxMountContext=""
Oct 30 12:57:04.306907 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.306633    4943 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4817ec63-f4d5-4420-9417-148346ff8bb5" volumeName="kubernetes.io/configmap/4817ec63-f4d5-4420-9417-148346ff8bb5-ovnkube-script-lib" seLinuxMountContext=""
Oct 30 12:57:04.306907 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.306641    4943 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a4d58318-42ba-4a9e-9c74-43fe90baa4a7" volumeName="kubernetes.io/projected/a4d58318-42ba-4a9e-9c74-43fe90baa4a7-kube-api-access-6sjmw" seLinuxMountContext=""
Oct 30 12:57:04.306907 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.306648    4943 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="23297e74-48f9-47b7-bb5a-80189bdbf2c4" volumeName="kubernetes.io/configmap/23297e74-48f9-47b7-bb5a-80189bdbf2c4-samples-operator-config" seLinuxMountContext=""
Oct 30 12:57:04.306907 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.306656    4943 reconstruct.go:97] "Volume reconstruction finished"
Oct 30 12:57:04.306907 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.306661    4943 reconciler.go:29] "Reconciler: start to sync state"
Oct 30 12:57:04.323770 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.323745    4943 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-8fqfr_d8930ee7-0ae0-48b5-8ce3-cb8820355cef/console-operator/1.log"
Oct 30 12:57:04.326443 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.326425    4943 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-8fqfr_d8930ee7-0ae0-48b5-8ce3-cb8820355cef/console-operator/1.log"
Oct 30 12:57:04.340656 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.340623    4943 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Oct 30 12:57:04.342458 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.342438    4943 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Oct 30 12:57:04.342546 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.342462    4943 status_manager.go:249] "Starting to sync pod status with apiserver"
Oct 30 12:57:04.342546 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.342478    4943 kubelet.go:2521] "Starting kubelet main sync loop"
Oct 30 12:57:04.342546 ip-10-0-142-84 kubenswrapper[4943]: E1030 12:57:04.342518    4943 kubelet.go:2545] "Skipping pod synchronization" err="[container runtime status check may not have completed yet, PLEG is not healthy: pleg has yet to be successful]"
Oct 30 12:57:04.358345 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.358319    4943 reflector.go:446] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 12:57:04.358532 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.358511    4943 reconstruct.go:205] "DevicePaths of reconstructed volumes updated"
Oct 30 12:57:04.358669 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.358544    4943 reflector.go:446] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 12:57:04.359908 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.359883    4943 generic.go:356] "Generic (PLEG): container finished" podID="9962cf52-bd50-4a97-a699-4da03085eb81" containerID="f2f38a511535e38fe136cff8ceee2c1c7807609526121506d357f18d9af124a6" exitCode=0
Oct 30 12:57:04.359908 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.359904    4943 generic.go:356] "Generic (PLEG): container finished" podID="9962cf52-bd50-4a97-a699-4da03085eb81" containerID="537f45f4a3e8b6659aa72e92aef20c8f0056d277c3831c358ed263d8d2c7629e" exitCode=0
Oct 30 12:57:04.359908 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.359910    4943 generic.go:356] "Generic (PLEG): container finished" podID="9962cf52-bd50-4a97-a699-4da03085eb81" containerID="7e6aed8f682a0a1e80249063c69ade97107d96da9122079edaa214cb269d3359" exitCode=0
Oct 30 12:57:04.360091 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.359918    4943 generic.go:356] "Generic (PLEG): container finished" podID="9962cf52-bd50-4a97-a699-4da03085eb81" containerID="caf5c243de766fef8c22ec79680a8f66b98b27dd428288c4a1c2a7d6ddbaf20d" exitCode=0
Oct 30 12:57:04.360091 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.359924    4943 generic.go:356] "Generic (PLEG): container finished" podID="9962cf52-bd50-4a97-a699-4da03085eb81" containerID="afb2509b01924c7b14517f5a1a969d41bb17933bce75a633ed981515cf663afc" exitCode=0
Oct 30 12:57:04.360091 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.359929    4943 generic.go:356] "Generic (PLEG): container finished" podID="9962cf52-bd50-4a97-a699-4da03085eb81" containerID="72b296697f032c3f34defef4af30184863991ef5228a2a1a619ee56ef45f1241" exitCode=0
Oct 30 12:57:04.366228 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.366191    4943 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-8fqfr_d8930ee7-0ae0-48b5-8ce3-cb8820355cef/console-operator/1.log"
Oct 30 12:57:04.366328 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.366242    4943 generic.go:356] "Generic (PLEG): container finished" podID="d8930ee7-0ae0-48b5-8ce3-cb8820355cef" containerID="0d08c313aba902fdd2b8020553685be737434486e076384118441c7db6d44b42" exitCode=255
Oct 30 12:57:04.371140 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.371119    4943 generic.go:356] "Generic (PLEG): container finished" podID="6dab6bee2c58785dc24bb59bb1408683" containerID="3e0d219cdbabee434c71ad7be165744aeaddbbf7941f509aeff6755af5119382" exitCode=0
Oct 30 12:57:04.372386 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.372367    4943 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 12:57:04.372761 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.372746    4943 reflector.go:446] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 12:57:04.395691 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.395673    4943 manager.go:324] Recovery completed
Oct 30 12:57:04.414802 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.414750    4943 cpu_manager.go:226] "Starting" policy="none"
Oct 30 12:57:04.414802 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.414763    4943 cpu_manager.go:227] "Reconciling" reconcilePeriod="10s"
Oct 30 12:57:04.414802 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.414778    4943 state_mem.go:41] "Initialized" logger="CPUManager state checkpoint.CPUManager state memory"
Oct 30 12:57:04.414991 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.414883    4943 state_mem.go:94] "Updated default CPUSet" logger="CPUManager state checkpoint.CPUManager state memory" cpuSet=""
Oct 30 12:57:04.414991 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.414893    4943 state_mem.go:102] "Updated CPUSet assignments" logger="CPUManager state checkpoint.CPUManager state memory" assignments={}
Oct 30 12:57:04.414991 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.414909    4943 state_checkpoint.go:142] "restored state from checkpoint" logger="CPUManager state checkpoint"
Oct 30 12:57:04.414991 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.414914    4943 state_checkpoint.go:143] "defaultCPUSet" logger="CPUManager state checkpoint" defaultCpuSet=""
Oct 30 12:57:04.414991 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.414919    4943 policy_none.go:50] "Start"
Oct 30 12:57:04.414991 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.414928    4943 memory_manager.go:187] "Starting memorymanager" policy="None"
Oct 30 12:57:04.414991 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.414937    4943 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Oct 30 12:57:04.415180 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.415052    4943 state_mem.go:77] "Updated machine memory state" logger="Memory Manager state checkpoint"
Oct 30 12:57:04.415180 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.415060    4943 state_checkpoint.go:85] "State checkpoint: restored state from checkpoint" logger="Memory Manager state checkpoint"
Oct 30 12:57:04.415180 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.415065    4943 policy_none.go:44] "Start"
Oct 30 12:57:04.419230 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.419204    4943 node_container_manager_linux.go:130] "Enforcing system reserved compressible on cgroup" cgroupName="/system.slice" limits={"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"}
Oct 30 12:57:04.421808 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.421795    4943 manager.go:341] "Starting Device Plugin manager"
Oct 30 12:57:04.422242 ip-10-0-142-84 kubenswrapper[4943]: E1030 12:57:04.421819    4943 manager.go:525] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Oct 30 12:57:04.422242 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.422242    4943 server.go:87] "Starting device plugin registration server"
Oct 30 12:57:04.422513 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.422500    4943 eviction_manager.go:194] "Eviction manager: starting control loop"
Oct 30 12:57:04.422568 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.422513    4943 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Oct 30 12:57:04.422649 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.422620    4943 plugin_watcher.go:53] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Oct 30 12:57:04.422778 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.422735    4943 plugin_watcher.go:201] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Oct 30 12:57:04.422885 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.422784    4943 plugin_manager.go:119] "The desired_state_of_world populator (plugin watcher) starts"
Oct 30 12:57:04.422885 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.422791    4943 plugin_manager.go:121] "Starting Kubelet Plugin Manager"
Oct 30 12:57:04.422972 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.422821    4943 reconciler.go:167] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-10-30T12:57:04.422772376Z","UUID":"a71cedff-876b-44b8-8944-8456283d8de3","Handler":null,"Name":"","Endpoint":""}
Oct 30 12:57:04.425291 ip-10-0-142-84 kubenswrapper[4943]: E1030 12:57:04.424246    4943 eviction_manager.go:272] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Oct 30 12:57:04.427076 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.427056    4943 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Oct 30 12:57:04.427163 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.427111    4943 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Oct 30 12:57:04.442608 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.442573    4943 kubelet.go:2608] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-84.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-142-84.ec2.internal"]
Oct 30 12:57:04.443142 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.443128    4943 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-84.ec2.internal"
Oct 30 12:57:04.443289 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.443120    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-84.ec2.internal" event={"ID":"6dab6bee2c58785dc24bb59bb1408683","Type":"ContainerStarted","Data":"39edb1d73c7239f611e2348acaf1783d095ee48b8a070c9406531ebc5b9e703d"}
Oct 30 12:57:04.443379 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.443306    4943 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-84.ec2.internal"
Oct 30 12:57:04.443379 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.443306    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-84.ec2.internal" event={"ID":"6dab6bee2c58785dc24bb59bb1408683","Type":"ContainerDied","Data":"3e0d219cdbabee434c71ad7be165744aeaddbbf7941f509aeff6755af5119382"}
Oct 30 12:57:04.443379 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.443334    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-84.ec2.internal" event={"ID":"6dab6bee2c58785dc24bb59bb1408683","Type":"ContainerStarted","Data":"afc6d32b630af29555dbee82da400929a106c898110d62a6e248dac42fa99dd1"}
Oct 30 12:57:04.443379 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.443366    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-84.ec2.internal" event={"ID":"0c9cf3ec0e3ab50517e6f80b07fb5fb2","Type":"ContainerStarted","Data":"1c55f0b1d73742e538f35231d27757d818ca3987270e1134eaafd83ed0bbe70c"}
Oct 30 12:57:04.443379 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.443381    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-84.ec2.internal" event={"ID":"0c9cf3ec0e3ab50517e6f80b07fb5fb2","Type":"ContainerStarted","Data":"56df6b59efaeea83af00f4daa02105b051a8b360001d76963a79837c1727270e"}
Oct 30 12:57:04.469913 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.469891    4943 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 30 12:57:04.470048 ip-10-0-142-84 kubenswrapper[4943]: E1030 12:57:04.469934    4943 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-apiserver-proxy-ip-10-0-142-84.ec2.internal\" already exists" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-84.ec2.internal"
Oct 30 12:57:04.470436 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.470424    4943 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 30 12:57:04.470476 ip-10-0-142-84 kubenswrapper[4943]: E1030 12:57:04.470451    4943 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-rbac-proxy-crio-ip-10-0-142-84.ec2.internal\" already exists" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-84.ec2.internal"
Oct 30 12:57:04.523112 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.523092    4943 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 12:57:04.525289 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.525274    4943 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-84.ec2.internal" event="NodeHasSufficientMemory"
Oct 30 12:57:04.525365 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.525304    4943 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-84.ec2.internal" event="NodeHasNoDiskPressure"
Oct 30 12:57:04.525365 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.525341    4943 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-84.ec2.internal" event="NodeHasSufficientPID"
Oct 30 12:57:04.525472 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.525396    4943 kubelet_node_status.go:77] "Attempting to register node" node="ip-10-0-142-84.ec2.internal"
Oct 30 12:57:04.559782 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.559762    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/6dab6bee2c58785dc24bb59bb1408683-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-84.ec2.internal\" (UID: \"6dab6bee2c58785dc24bb59bb1408683\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-84.ec2.internal"
Oct 30 12:57:04.559850 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.559796    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6dab6bee2c58785dc24bb59bb1408683-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-84.ec2.internal\" (UID: \"6dab6bee2c58785dc24bb59bb1408683\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-84.ec2.internal"
Oct 30 12:57:04.559850 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.559815    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/0c9cf3ec0e3ab50517e6f80b07fb5fb2-config\") pod \"kube-apiserver-proxy-ip-10-0-142-84.ec2.internal\" (UID: \"0c9cf3ec0e3ab50517e6f80b07fb5fb2\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-84.ec2.internal"
Oct 30 12:57:04.562703 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.562687    4943 kubelet_node_status.go:126] "Node was previously registered" node="ip-10-0-142-84.ec2.internal"
Oct 30 12:57:04.562788 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.562751    4943 kubelet_node_status.go:80] "Successfully registered node" node="ip-10-0-142-84.ec2.internal"
Oct 30 12:57:04.660805 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.660777    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/6dab6bee2c58785dc24bb59bb1408683-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-84.ec2.internal\" (UID: \"6dab6bee2c58785dc24bb59bb1408683\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-84.ec2.internal"
Oct 30 12:57:04.660885 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.660805    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6dab6bee2c58785dc24bb59bb1408683-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-84.ec2.internal\" (UID: \"6dab6bee2c58785dc24bb59bb1408683\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-84.ec2.internal"
Oct 30 12:57:04.660885 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.660825    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/0c9cf3ec0e3ab50517e6f80b07fb5fb2-config\") pod \"kube-apiserver-proxy-ip-10-0-142-84.ec2.internal\" (UID: \"0c9cf3ec0e3ab50517e6f80b07fb5fb2\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-84.ec2.internal"
Oct 30 12:57:04.660959 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.660882    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/0c9cf3ec0e3ab50517e6f80b07fb5fb2-config\") pod \"kube-apiserver-proxy-ip-10-0-142-84.ec2.internal\" (UID: \"0c9cf3ec0e3ab50517e6f80b07fb5fb2\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-84.ec2.internal"
Oct 30 12:57:04.660959 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.660908    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/6dab6bee2c58785dc24bb59bb1408683-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-84.ec2.internal\" (UID: \"6dab6bee2c58785dc24bb59bb1408683\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-84.ec2.internal"
Oct 30 12:57:04.660959 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:04.660882    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6dab6bee2c58785dc24bb59bb1408683-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-84.ec2.internal\" (UID: \"6dab6bee2c58785dc24bb59bb1408683\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-84.ec2.internal"
Oct 30 12:57:05.288908 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.288879    4943 apiserver.go:52] "Watching apiserver"
Oct 30 12:57:05.363628 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.363592    4943 reflector.go:446] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Oct 30 12:57:05.364334 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.364307    4943 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8bnjr","openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-7dtcd","kube-system/kas-connection-checker-685895c7c9-k4v7j","openshift-network-console/networking-console-plugin-86c478bb88-vsqgb","kube-system/kube-apiserver-proxy-ip-10-0-142-84.ec2.internal","openshift-ingress/router-default-649bdc978f-lkpqz","openshift-insights/insights-runtime-extractor-knqz6","openshift-network-diagnostics/network-check-source-6f78d59f99-vdwgx","openshift-insights/insights-operator-7c4986d449-7cfvt","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-84.ec2.internal","openshift-kube-storage-version-migrator/migrator-66df5d496b-cfhqz","openshift-ovn-kubernetes/ovnkube-node-7rwcq","openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-plmkt","kube-system/kas-connection-checker-685895c7c9-4zl6m","openshift-image-registry/node-ca-h8q28","openshift-image-registry/image-registry-69c87564f5-n6wjk","openshift-multus/multus-additional-cni-plugins-m4p7x","openshift-multus/multus-2l8x5","openshift-service-ca-operator/service-ca-operator-544d749c8c-h42v2","kube-system/konnectivity-agent-zth9k","openshift-image-registry/image-registry-77c6ff895d-vw6c6","openshift-network-diagnostics/network-check-target-l5mdf","kube-system/kas-connection-checker-685895c7c9-h7brb","openshift-network-operator/iptables-alerter-kb9lc","openshift-dns/dns-default-g2ckh","openshift-console-operator/console-operator-7cd56b8dbc-8fqfr","openshift-dns/node-resolver-4lfbk","openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-xtfpg","openshift-multus/network-metrics-daemon-p7tv6","openshift-monitoring/cluster-monitoring-operator-884576c57-2sfj9","openshift-cluster-node-tuning-operator/tuned-8mfkz","openshift-ingress-canary/ingress-canary-bdf5s","kube-system/global-pull-secret-syncer-rpdcd"]
Oct 30 12:57:05.367712 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.367683    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-884576c57-2sfj9"
Oct 30 12:57:05.368306 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.368253    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-7dtcd"
Oct 30 12:57:05.369144 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.369128    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-649bdc978f-lkpqz"
Oct 30 12:57:05.369509 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.369484    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-77c6ff895d-vw6c6"
Oct 30 12:57:05.371547 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.371530    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-86c478bb88-vsqgb"
Oct 30 12:57:05.373726 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.373711    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p7tv6"
Oct 30 12:57:05.375948 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.375932    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-69c87564f5-n6wjk"
Oct 30 12:57:05.377533 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.377515    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-g2ckh"
Oct 30 12:57:05.378698 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.378682    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-knqz6"
Oct 30 12:57:05.382382 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.382369    4943 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"openshift-service-ca.crt\""
Oct 30 12:57:05.382438 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.382375    4943 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"kube-root-ca.crt\""
Oct 30 12:57:05.384369 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.384355    4943 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\""
Oct 30 12:57:05.384738 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.384718    4943 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\""
Oct 30 12:57:05.385886 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.385872    4943 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Oct 30 12:57:05.386022 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.386009    4943 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\""
Oct 30 12:57:05.386526 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.386506    4943 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\""
Oct 30 12:57:05.386944 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.386924    4943 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-config\""
Oct 30 12:57:05.392652 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.392634    4943 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-84.ec2.internal"
Oct 30 12:57:05.392815 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.392801    4943 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-84.ec2.internal"
Oct 30 12:57:05.401396 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.401374    4943 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\""
Oct 30 12:57:05.401396 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.401381    4943 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\""
Oct 30 12:57:05.401659 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.401411    4943 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\""
Oct 30 12:57:05.401659 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.401435    4943 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\""
Oct 30 12:57:05.401659 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.401440    4943 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\""
Oct 30 12:57:05.401659 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.401458    4943 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Oct 30 12:57:05.401659 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.401495    4943 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Oct 30 12:57:05.401659 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.401517    4943 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Oct 30 12:57:05.401659 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.401538    4943 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Oct 30 12:57:05.401659 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.401545    4943 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Oct 30 12:57:05.401659 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.401557    4943 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\""
Oct 30 12:57:05.401659 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.401555    4943 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\""
Oct 30 12:57:05.401659 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.401594    4943 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\""
Oct 30 12:57:05.401659 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.401616    4943 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\""
Oct 30 12:57:05.401659 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.401655    4943 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Oct 30 12:57:05.402543 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.401772    4943 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Oct 30 12:57:05.402543 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.401988    4943 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\""
Oct 30 12:57:05.402543 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.402018    4943 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Oct 30 12:57:05.402543 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.402070    4943 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\""
Oct 30 12:57:05.402543 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.402279    4943 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Oct 30 12:57:05.402543 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.402287    4943 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\""
Oct 30 12:57:05.402543 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.402326    4943 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
Oct 30 12:57:05.402543 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.402331    4943 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Oct 30 12:57:05.402543 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.402369    4943 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Oct 30 12:57:05.402543 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.402337    4943 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Oct 30 12:57:05.402543 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.402345    4943 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Oct 30 12:57:05.402543 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.402382    4943 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Oct 30 12:57:05.402543 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.402339    4943 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Oct 30 12:57:05.402543 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.402453    4943 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\""
Oct 30 12:57:05.402543 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.402475    4943 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\""
Oct 30 12:57:05.405053 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.405039    4943 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Oct 30 12:57:05.415801 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.415781    4943 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Oct 30 12:57:05.415900 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.415837    4943 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Oct 30 12:57:05.415900 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.415836    4943 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Oct 30 12:57:05.415900 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.415857    4943 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Oct 30 12:57:05.415900 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.415887    4943 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Oct 30 12:57:05.416135 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.415948    4943 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Oct 30 12:57:05.416135 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.415957    4943 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Oct 30 12:57:05.416135 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.415964    4943 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Oct 30 12:57:05.416135 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.415963    4943 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Oct 30 12:57:05.416135 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.416034    4943 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Oct 30 12:57:05.416135 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.416041    4943 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Oct 30 12:57:05.416135 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.416076    4943 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Oct 30 12:57:05.416135 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.416063    4943 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Oct 30 12:57:05.416135 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.416087    4943 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Oct 30 12:57:05.416135 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.416120    4943 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Oct 30 12:57:05.416629 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.416191    4943 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\""
Oct 30 12:57:05.416629 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.416199    4943 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Oct 30 12:57:05.416629 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.416202    4943 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Oct 30 12:57:05.416629 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.416213    4943 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\""
Oct 30 12:57:05.416629 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.416218    4943 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Oct 30 12:57:05.416629 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.416231    4943 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Oct 30 12:57:05.416629 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.416232    4943 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Oct 30 12:57:05.416629 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.416244    4943 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Oct 30 12:57:05.416629 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.416252    4943 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Oct 30 12:57:05.416629 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.416254    4943 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Oct 30 12:57:05.416629 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.416240    4943 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Oct 30 12:57:05.416629 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.416293    4943 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Oct 30 12:57:05.416629 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.416324    4943 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\""
Oct 30 12:57:05.416629 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.416345    4943 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Oct 30 12:57:05.416629 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.416514    4943 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Oct 30 12:57:05.417627 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.417612    4943 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 30 12:57:05.417683 ip-10-0-142-84 kubenswrapper[4943]: E1030 12:57:05.417649    4943 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-apiserver-proxy-ip-10-0-142-84.ec2.internal\" already exists" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-84.ec2.internal"
Oct 30 12:57:05.418472 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.418457    4943 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 30 12:57:05.418542 ip-10-0-142-84 kubenswrapper[4943]: E1030 12:57:05.418501    4943 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-rbac-proxy-crio-ip-10-0-142-84.ec2.internal\" already exists" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-84.ec2.internal"
Oct 30 12:57:05.464835 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.464816    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/55fa4842-3eb4-4e52-a4fc-18876ca409d5-crio-socket\") pod \"insights-runtime-extractor-knqz6\" (UID: \"55fa4842-3eb4-4e52-a4fc-18876ca409d5\") " pod="openshift-insights/insights-runtime-extractor-knqz6"
Oct 30 12:57:05.464937 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.464844    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/55fa4842-3eb4-4e52-a4fc-18876ca409d5-data-volume\") pod \"insights-runtime-extractor-knqz6\" (UID: \"55fa4842-3eb4-4e52-a4fc-18876ca409d5\") " pod="openshift-insights/insights-runtime-extractor-knqz6"
Oct 30 12:57:05.464937 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.464859    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/4817ec63-f4d5-4420-9417-148346ff8bb5-run-systemd\") pod \"ovnkube-node-7rwcq\" (UID: \"4817ec63-f4d5-4420-9417-148346ff8bb5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7rwcq"
Oct 30 12:57:05.464937 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.464877    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qwmfv\" (UniqueName: \"kubernetes.io/projected/19b38c46-bfd4-4ed0-a764-f7b672f18cc6-kube-api-access-qwmfv\") pod \"cluster-monitoring-operator-884576c57-2sfj9\" (UID: \"19b38c46-bfd4-4ed0-a764-f7b672f18cc6\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-2sfj9"
Oct 30 12:57:05.464937 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.464901    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bd9e55af-64ad-4ffc-92c3-53b7bf654322-service-ca-bundle\") pod \"insights-operator-7c4986d449-7cfvt\" (UID: \"bd9e55af-64ad-4ffc-92c3-53b7bf654322\") " pod="openshift-insights/insights-operator-7c4986d449-7cfvt"
Oct 30 12:57:05.465053 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.464962    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e1f629e6-6a96-444f-8329-e71bb98c9029-service-ca-bundle\") pod \"router-default-649bdc978f-lkpqz\" (UID: \"e1f629e6-6a96-444f-8329-e71bb98c9029\") " pod="openshift-ingress/router-default-649bdc978f-lkpqz"
Oct 30 12:57:05.465053 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.464962    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/55fa4842-3eb4-4e52-a4fc-18876ca409d5-data-volume\") pod \"insights-runtime-extractor-knqz6\" (UID: \"55fa4842-3eb4-4e52-a4fc-18876ca409d5\") " pod="openshift-insights/insights-runtime-extractor-knqz6"
Oct 30 12:57:05.465128 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.465114    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bd9e55af-64ad-4ffc-92c3-53b7bf654322-service-ca-bundle\") pod \"insights-operator-7c4986d449-7cfvt\" (UID: \"bd9e55af-64ad-4ffc-92c3-53b7bf654322\") " pod="openshift-insights/insights-operator-7c4986d449-7cfvt"
Oct 30 12:57:05.465159 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.465127    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/e1f629e6-6a96-444f-8329-e71bb98c9029-stats-auth\") pod \"router-default-649bdc978f-lkpqz\" (UID: \"e1f629e6-6a96-444f-8329-e71bb98c9029\") " pod="openshift-ingress/router-default-649bdc978f-lkpqz"
Oct 30 12:57:05.465194 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.465179    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e1f629e6-6a96-444f-8329-e71bb98c9029-metrics-certs\") pod \"router-default-649bdc978f-lkpqz\" (UID: \"e1f629e6-6a96-444f-8329-e71bb98c9029\") " pod="openshift-ingress/router-default-649bdc978f-lkpqz"
Oct 30 12:57:05.465319 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.465301    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-phkm5\" (UniqueName: \"kubernetes.io/projected/e1f629e6-6a96-444f-8329-e71bb98c9029-kube-api-access-phkm5\") pod \"router-default-649bdc978f-lkpqz\" (UID: \"e1f629e6-6a96-444f-8329-e71bb98c9029\") " pod="openshift-ingress/router-default-649bdc978f-lkpqz"
Oct 30 12:57:05.465362 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.465341    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l8pqr\" (UniqueName: \"kubernetes.io/projected/b86e8e45-3c2d-46b6-9ade-d390b56e4a75-kube-api-access-l8pqr\") pod \"image-registry-69c87564f5-n6wjk\" (UID: \"b86e8e45-3c2d-46b6-9ade-d390b56e4a75\") " pod="openshift-image-registry/image-registry-69c87564f5-n6wjk"
Oct 30 12:57:05.465362 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.465357    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9962cf52-bd50-4a97-a699-4da03085eb81-cni-binary-copy\") pod \"multus-additional-cni-plugins-m4p7x\" (UID: \"9962cf52-bd50-4a97-a699-4da03085eb81\") " pod="openshift-multus/multus-additional-cni-plugins-m4p7x"
Oct 30 12:57:05.465424 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.465374    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/d8930ee7-0ae0-48b5-8ce3-cb8820355cef-config\") pod \"console-operator-7cd56b8dbc-8fqfr\" (UID: \"d8930ee7-0ae0-48b5-8ce3-cb8820355cef\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-8fqfr"
Oct 30 12:57:05.465424 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.465390    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/4817ec63-f4d5-4420-9417-148346ff8bb5-host-cni-netd\") pod \"ovnkube-node-7rwcq\" (UID: \"4817ec63-f4d5-4420-9417-148346ff8bb5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7rwcq"
Oct 30 12:57:05.465424 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.465408    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-swxqr\" (UniqueName: \"kubernetes.io/projected/0905782c-26ad-4dc8-93cf-d6f211ea7af5-kube-api-access-swxqr\") pod \"kas-connection-checker-685895c7c9-k4v7j\" (UID: \"0905782c-26ad-4dc8-93cf-d6f211ea7af5\") " pod="kube-system/kas-connection-checker-685895c7c9-k4v7j"
Oct 30 12:57:05.465548 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.465425    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/56ff7ccd-573e-4bb2-a2b7-199a0346ed9f-run\") pod \"tuned-8mfkz\" (UID: \"56ff7ccd-573e-4bb2-a2b7-199a0346ed9f\") " pod="openshift-cluster-node-tuning-operator/tuned-8mfkz"
Oct 30 12:57:05.465548 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.465433    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/e1f629e6-6a96-444f-8329-e71bb98c9029-stats-auth\") pod \"router-default-649bdc978f-lkpqz\" (UID: \"e1f629e6-6a96-444f-8329-e71bb98c9029\") " pod="openshift-ingress/router-default-649bdc978f-lkpqz"
Oct 30 12:57:05.465548 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.465439    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9a20c40d-f5f4-4d96-85b6-be69a6cd1828-host-var-lib-kubelet\") pod \"multus-2l8x5\" (UID: \"9a20c40d-f5f4-4d96-85b6-be69a6cd1828\") " pod="openshift-multus/multus-2l8x5"
Oct 30 12:57:05.465654 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.465583    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/85f4c399-0089-4399-86e2-069834f89286-config\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-plmkt\" (UID: \"85f4c399-0089-4399-86e2-069834f89286\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-plmkt"
Oct 30 12:57:05.465724 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.465707    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qxmjj\" (UniqueName: \"kubernetes.io/projected/fcf99170-f5cf-4987-97bd-1c30b1b37a35-kube-api-access-qxmjj\") pod \"dns-default-g2ckh\" (UID: \"fcf99170-f5cf-4987-97bd-1c30b1b37a35\") " pod="openshift-dns/dns-default-g2ckh"
Oct 30 12:57:05.465781 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.465720    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/d8930ee7-0ae0-48b5-8ce3-cb8820355cef-config\") pod \"console-operator-7cd56b8dbc-8fqfr\" (UID: \"d8930ee7-0ae0-48b5-8ce3-cb8820355cef\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-8fqfr"
Oct 30 12:57:05.465781 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.465742    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bd9e55af-64ad-4ffc-92c3-53b7bf654322-tmp\") pod \"insights-operator-7c4986d449-7cfvt\" (UID: \"bd9e55af-64ad-4ffc-92c3-53b7bf654322\") " pod="openshift-insights/insights-operator-7c4986d449-7cfvt"
Oct 30 12:57:05.465876 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.465827    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9962cf52-bd50-4a97-a699-4da03085eb81-cni-binary-copy\") pod \"multus-additional-cni-plugins-m4p7x\" (UID: \"9962cf52-bd50-4a97-a699-4da03085eb81\") " pod="openshift-multus/multus-additional-cni-plugins-m4p7x"
Oct 30 12:57:05.465876 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.465836    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bd9e55af-64ad-4ffc-92c3-53b7bf654322-trusted-ca-bundle\") pod \"insights-operator-7c4986d449-7cfvt\" (UID: \"bd9e55af-64ad-4ffc-92c3-53b7bf654322\") " pod="openshift-insights/insights-operator-7c4986d449-7cfvt"
Oct 30 12:57:05.465977 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.465875    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-54kft\" (UniqueName: \"kubernetes.io/projected/1c405568-cb88-4485-ae2c-3614cc31ced6-kube-api-access-54kft\") pod \"image-registry-77c6ff895d-vw6c6\" (UID: \"1c405568-cb88-4485-ae2c-3614cc31ced6\") " pod="openshift-image-registry/image-registry-77c6ff895d-vw6c6"
Oct 30 12:57:05.465977 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.465893    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hfnqv\" (UniqueName: \"kubernetes.io/projected/27c55f84-4e9a-44de-94ea-02258d6bc747-kube-api-access-hfnqv\") pod \"service-ca-operator-544d749c8c-h42v2\" (UID: \"27c55f84-4e9a-44de-94ea-02258d6bc747\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-h42v2"
Oct 30 12:57:05.465977 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.465890    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bd9e55af-64ad-4ffc-92c3-53b7bf654322-tmp\") pod \"insights-operator-7c4986d449-7cfvt\" (UID: \"bd9e55af-64ad-4ffc-92c3-53b7bf654322\") " pod="openshift-insights/insights-operator-7c4986d449-7cfvt"
Oct 30 12:57:05.465977 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.465890    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/85f4c399-0089-4399-86e2-069834f89286-config\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-plmkt\" (UID: \"85f4c399-0089-4399-86e2-069834f89286\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-plmkt"
Oct 30 12:57:05.465977 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.465913    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9a20c40d-f5f4-4d96-85b6-be69a6cd1828-host-run-netns\") pod \"multus-2l8x5\" (UID: \"9a20c40d-f5f4-4d96-85b6-be69a6cd1828\") " pod="openshift-multus/multus-2l8x5"
Oct 30 12:57:05.465977 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.465953    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9962cf52-bd50-4a97-a699-4da03085eb81-system-cni-dir\") pod \"multus-additional-cni-plugins-m4p7x\" (UID: \"9962cf52-bd50-4a97-a699-4da03085eb81\") " pod="openshift-multus/multus-additional-cni-plugins-m4p7x"
Oct 30 12:57:05.466189 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.466005    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/a3ceb095-c75c-43b3-9de5-7dda644eddb1-kubelet-dir\") pod \"aws-ebs-csi-driver-node-8bnjr\" (UID: \"a3ceb095-c75c-43b3-9de5-7dda644eddb1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8bnjr"
Oct 30 12:57:05.466189 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.466025    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4817ec63-f4d5-4420-9417-148346ff8bb5-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-7rwcq\" (UID: \"4817ec63-f4d5-4420-9417-148346ff8bb5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7rwcq"
Oct 30 12:57:05.466189 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.466067    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9a20c40d-f5f4-4d96-85b6-be69a6cd1828-cnibin\") pod \"multus-2l8x5\" (UID: \"9a20c40d-f5f4-4d96-85b6-be69a6cd1828\") " pod="openshift-multus/multus-2l8x5"
Oct 30 12:57:05.466189 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.466110    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9a20c40d-f5f4-4d96-85b6-be69a6cd1828-etc-kubernetes\") pod \"multus-2l8x5\" (UID: \"9a20c40d-f5f4-4d96-85b6-be69a6cd1828\") " pod="openshift-multus/multus-2l8x5"
Oct 30 12:57:05.466189 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.466150    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/55fa4842-3eb4-4e52-a4fc-18876ca409d5-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-knqz6\" (UID: \"55fa4842-3eb4-4e52-a4fc-18876ca409d5\") " pod="openshift-insights/insights-runtime-extractor-knqz6"
Oct 30 12:57:05.466189 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.466162    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bd9e55af-64ad-4ffc-92c3-53b7bf654322-trusted-ca-bundle\") pod \"insights-operator-7c4986d449-7cfvt\" (UID: \"bd9e55af-64ad-4ffc-92c3-53b7bf654322\") " pod="openshift-insights/insights-operator-7c4986d449-7cfvt"
Oct 30 12:57:05.466386 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.466188    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1c405568-cb88-4485-ae2c-3614cc31ced6-bound-sa-token\") pod \"image-registry-77c6ff895d-vw6c6\" (UID: \"1c405568-cb88-4485-ae2c-3614cc31ced6\") " pod="openshift-image-registry/image-registry-77c6ff895d-vw6c6"
Oct 30 12:57:05.466386 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.466226    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/9962cf52-bd50-4a97-a699-4da03085eb81-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-m4p7x\" (UID: \"9962cf52-bd50-4a97-a699-4da03085eb81\") " pod="openshift-multus/multus-additional-cni-plugins-m4p7x"
Oct 30 12:57:05.466386 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.466299    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/56ff7ccd-573e-4bb2-a2b7-199a0346ed9f-host\") pod \"tuned-8mfkz\" (UID: \"56ff7ccd-573e-4bb2-a2b7-199a0346ed9f\") " pod="openshift-cluster-node-tuning-operator/tuned-8mfkz"
Oct 30 12:57:05.466386 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.466329    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/85f4c399-0089-4399-86e2-069834f89286-serving-cert\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-plmkt\" (UID: \"85f4c399-0089-4399-86e2-069834f89286\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-plmkt"
Oct 30 12:57:05.466386 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.466354    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ac1dc607-3072-4dbc-87e9-db316bf42dff-host-slash\") pod \"iptables-alerter-kb9lc\" (UID: \"ac1dc607-3072-4dbc-87e9-db316bf42dff\") " pod="openshift-network-operator/iptables-alerter-kb9lc"
Oct 30 12:57:05.466546 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.466390    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/b86e8e45-3c2d-46b6-9ade-d390b56e4a75-trusted-ca\") pod \"image-registry-69c87564f5-n6wjk\" (UID: \"b86e8e45-3c2d-46b6-9ade-d390b56e4a75\") " pod="openshift-image-registry/image-registry-69c87564f5-n6wjk"
Oct 30 12:57:05.466546 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.466425    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/9962cf52-bd50-4a97-a699-4da03085eb81-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-m4p7x\" (UID: \"9962cf52-bd50-4a97-a699-4da03085eb81\") " pod="openshift-multus/multus-additional-cni-plugins-m4p7x"
Oct 30 12:57:05.466546 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.466489    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b86e8e45-3c2d-46b6-9ade-d390b56e4a75-bound-sa-token\") pod \"image-registry-69c87564f5-n6wjk\" (UID: \"b86e8e45-3c2d-46b6-9ade-d390b56e4a75\") " pod="openshift-image-registry/image-registry-69c87564f5-n6wjk"
Oct 30 12:57:05.466546 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.466516    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7rwfx\" (UniqueName: \"kubernetes.io/projected/4817ec63-f4d5-4420-9417-148346ff8bb5-kube-api-access-7rwfx\") pod \"ovnkube-node-7rwcq\" (UID: \"4817ec63-f4d5-4420-9417-148346ff8bb5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7rwcq"
Oct 30 12:57:05.466734 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.466550    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/56ff7ccd-573e-4bb2-a2b7-199a0346ed9f-etc-modprobe-d\") pod \"tuned-8mfkz\" (UID: \"56ff7ccd-573e-4bb2-a2b7-199a0346ed9f\") " pod="openshift-cluster-node-tuning-operator/tuned-8mfkz"
Oct 30 12:57:05.466734 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.466575    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/4883c26d-0877-4870-b0ee-406f5bfa8cf4-nginx-conf\") pod \"networking-console-plugin-86c478bb88-vsqgb\" (UID: \"4883c26d-0877-4870-b0ee-406f5bfa8cf4\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-vsqgb"
Oct 30 12:57:05.466734 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.466604    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/d8930ee7-0ae0-48b5-8ce3-cb8820355cef-serving-cert\") pod \"console-operator-7cd56b8dbc-8fqfr\" (UID: \"d8930ee7-0ae0-48b5-8ce3-cb8820355cef\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-8fqfr"
Oct 30 12:57:05.466734 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.466631    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d8930ee7-0ae0-48b5-8ce3-cb8820355cef-trusted-ca\") pod \"console-operator-7cd56b8dbc-8fqfr\" (UID: \"d8930ee7-0ae0-48b5-8ce3-cb8820355cef\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-8fqfr"
Oct 30 12:57:05.466734 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.466661    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/1c405568-cb88-4485-ae2c-3614cc31ced6-image-registry-private-configuration\") pod \"image-registry-77c6ff895d-vw6c6\" (UID: \"1c405568-cb88-4485-ae2c-3614cc31ced6\") " pod="openshift-image-registry/image-registry-77c6ff895d-vw6c6"
Oct 30 12:57:05.466734 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.466695    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/1c405568-cb88-4485-ae2c-3614cc31ced6-registry-certificates\") pod \"image-registry-77c6ff895d-vw6c6\" (UID: \"1c405568-cb88-4485-ae2c-3614cc31ced6\") " pod="openshift-image-registry/image-registry-77c6ff895d-vw6c6"
Oct 30 12:57:05.466734 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.466728    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/4817ec63-f4d5-4420-9417-148346ff8bb5-log-socket\") pod \"ovnkube-node-7rwcq\" (UID: \"4817ec63-f4d5-4420-9417-148346ff8bb5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7rwcq"
Oct 30 12:57:05.467040 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.466757    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/f660a256-a12d-4a4a-823d-4b0ff5dcb3da-kubelet-config\") pod \"global-pull-secret-syncer-rpdcd\" (UID: \"f660a256-a12d-4a4a-823d-4b0ff5dcb3da\") " pod="kube-system/global-pull-secret-syncer-rpdcd"
Oct 30 12:57:05.467040 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.466784    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9a20c40d-f5f4-4d96-85b6-be69a6cd1828-system-cni-dir\") pod \"multus-2l8x5\" (UID: \"9a20c40d-f5f4-4d96-85b6-be69a6cd1828\") " pod="openshift-multus/multus-2l8x5"
Oct 30 12:57:05.467040 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.466807    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/d8930ee7-0ae0-48b5-8ce3-cb8820355cef-serving-cert\") pod \"console-operator-7cd56b8dbc-8fqfr\" (UID: \"d8930ee7-0ae0-48b5-8ce3-cb8820355cef\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-8fqfr"
Oct 30 12:57:05.467040 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.466806    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/4883c26d-0877-4870-b0ee-406f5bfa8cf4-nginx-conf\") pod \"networking-console-plugin-86c478bb88-vsqgb\" (UID: \"4883c26d-0877-4870-b0ee-406f5bfa8cf4\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-vsqgb"
Oct 30 12:57:05.467040 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.466837    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/27c55f84-4e9a-44de-94ea-02258d6bc747-serving-cert\") pod \"service-ca-operator-544d749c8c-h42v2\" (UID: \"27c55f84-4e9a-44de-94ea-02258d6bc747\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-h42v2"
Oct 30 12:57:05.467040 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.466845    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/1c405568-cb88-4485-ae2c-3614cc31ced6-image-registry-private-configuration\") pod \"image-registry-77c6ff895d-vw6c6\" (UID: \"1c405568-cb88-4485-ae2c-3614cc31ced6\") " pod="openshift-image-registry/image-registry-77c6ff895d-vw6c6"
Oct 30 12:57:05.467040 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.466890    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/27c55f84-4e9a-44de-94ea-02258d6bc747-config\") pod \"service-ca-operator-544d749c8c-h42v2\" (UID: \"27c55f84-4e9a-44de-94ea-02258d6bc747\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-h42v2"
Oct 30 12:57:05.467040 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.466967    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/56ff7ccd-573e-4bb2-a2b7-199a0346ed9f-etc-systemd\") pod \"tuned-8mfkz\" (UID: \"56ff7ccd-573e-4bb2-a2b7-199a0346ed9f\") " pod="openshift-cluster-node-tuning-operator/tuned-8mfkz"
Oct 30 12:57:05.467040 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.466990    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/1c405568-cb88-4485-ae2c-3614cc31ced6-registry-certificates\") pod \"image-registry-77c6ff895d-vw6c6\" (UID: \"1c405568-cb88-4485-ae2c-3614cc31ced6\") " pod="openshift-image-registry/image-registry-77c6ff895d-vw6c6"
Oct 30 12:57:05.467040 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.467019    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/23297e74-48f9-47b7-bb5a-80189bdbf2c4-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-7dtcd\" (UID: \"23297e74-48f9-47b7-bb5a-80189bdbf2c4\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-7dtcd"
Oct 30 12:57:05.467341 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.467315    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/51d01732-87a4-42b0-a6d5-a3fac689d7f6-agent-certs\") pod \"konnectivity-agent-zth9k\" (UID: \"51d01732-87a4-42b0-a6d5-a3fac689d7f6\") " pod="kube-system/konnectivity-agent-zth9k"
Oct 30 12:57:05.467387 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.467367    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j2mgz\" (UniqueName: \"kubernetes.io/projected/eca286e8-0b02-42af-9d15-1d8670b7362c-kube-api-access-j2mgz\") pod \"node-resolver-4lfbk\" (UID: \"eca286e8-0b02-42af-9d15-1d8670b7362c\") " pod="openshift-dns/node-resolver-4lfbk"
Oct 30 12:57:05.467434 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.467397    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/9962cf52-bd50-4a97-a699-4da03085eb81-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-m4p7x\" (UID: \"9962cf52-bd50-4a97-a699-4da03085eb81\") " pod="openshift-multus/multus-additional-cni-plugins-m4p7x"
Oct 30 12:57:05.467480 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.467432    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/4817ec63-f4d5-4420-9417-148346ff8bb5-ovnkube-config\") pod \"ovnkube-node-7rwcq\" (UID: \"4817ec63-f4d5-4420-9417-148346ff8bb5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7rwcq"
Oct 30 12:57:05.467480 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.467466    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-49z5c\" (UniqueName: \"kubernetes.io/projected/301eebcb-3d48-4eef-a0de-a1173e23ad65-kube-api-access-49z5c\") pod \"kas-connection-checker-685895c7c9-h7brb\" (UID: \"301eebcb-3d48-4eef-a0de-a1173e23ad65\") " pod="kube-system/kas-connection-checker-685895c7c9-h7brb"
Oct 30 12:57:05.467569 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.467503    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/e1f629e6-6a96-444f-8329-e71bb98c9029-default-certificate\") pod \"router-default-649bdc978f-lkpqz\" (UID: \"e1f629e6-6a96-444f-8329-e71bb98c9029\") " pod="openshift-ingress/router-default-649bdc978f-lkpqz"
Oct 30 12:57:05.467656 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.467637    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/27c55f84-4e9a-44de-94ea-02258d6bc747-config\") pod \"service-ca-operator-544d749c8c-h42v2\" (UID: \"27c55f84-4e9a-44de-94ea-02258d6bc747\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-h42v2"
Oct 30 12:57:05.467710 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.467651    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9962cf52-bd50-4a97-a699-4da03085eb81-tuning-conf-dir\") pod \"multus-additional-cni-plugins-m4p7x\" (UID: \"9962cf52-bd50-4a97-a699-4da03085eb81\") " pod="openshift-multus/multus-additional-cni-plugins-m4p7x"
Oct 30 12:57:05.467710 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.467664    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/9962cf52-bd50-4a97-a699-4da03085eb81-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-m4p7x\" (UID: \"9962cf52-bd50-4a97-a699-4da03085eb81\") " pod="openshift-multus/multus-additional-cni-plugins-m4p7x"
Oct 30 12:57:05.467710 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.467465    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/b86e8e45-3c2d-46b6-9ade-d390b56e4a75-trusted-ca\") pod \"image-registry-69c87564f5-n6wjk\" (UID: \"b86e8e45-3c2d-46b6-9ade-d390b56e4a75\") " pod="openshift-image-registry/image-registry-69c87564f5-n6wjk"
Oct 30 12:57:05.467710 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.467682    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9vr7r\" (UniqueName: \"kubernetes.io/projected/8d767e44-14df-4ea6-b391-bdf265773535-kube-api-access-9vr7r\") pod \"network-check-target-l5mdf\" (UID: \"8d767e44-14df-4ea6-b391-bdf265773535\") " pod="openshift-network-diagnostics/network-check-target-l5mdf"
Oct 30 12:57:05.467888 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.467730    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4817ec63-f4d5-4420-9417-148346ff8bb5-host-run-ovn-kubernetes\") pod \"ovnkube-node-7rwcq\" (UID: \"4817ec63-f4d5-4420-9417-148346ff8bb5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7rwcq"
Oct 30 12:57:05.467888 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.467766    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/9a20c40d-f5f4-4d96-85b6-be69a6cd1828-host-run-multus-certs\") pod \"multus-2l8x5\" (UID: \"9a20c40d-f5f4-4d96-85b6-be69a6cd1828\") " pod="openshift-multus/multus-2l8x5"
Oct 30 12:57:05.467986 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.467925    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d8930ee7-0ae0-48b5-8ce3-cb8820355cef-trusted-ca\") pod \"console-operator-7cd56b8dbc-8fqfr\" (UID: \"d8930ee7-0ae0-48b5-8ce3-cb8820355cef\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-8fqfr"
Oct 30 12:57:05.468076 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.468055    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/19b38c46-bfd4-4ed0-a764-f7b672f18cc6-telemetry-config\") pod \"cluster-monitoring-operator-884576c57-2sfj9\" (UID: \"19b38c46-bfd4-4ed0-a764-f7b672f18cc6\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-2sfj9"
Oct 30 12:57:05.468134 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.468108    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/bd9e55af-64ad-4ffc-92c3-53b7bf654322-snapshots\") pod \"insights-operator-7c4986d449-7cfvt\" (UID: \"bd9e55af-64ad-4ffc-92c3-53b7bf654322\") " pod="openshift-insights/insights-operator-7c4986d449-7cfvt"
Oct 30 12:57:05.468289 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.468255    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/4817ec63-f4d5-4420-9417-148346ff8bb5-ovnkube-config\") pod \"ovnkube-node-7rwcq\" (UID: \"4817ec63-f4d5-4420-9417-148346ff8bb5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7rwcq"
Oct 30 12:57:05.468398 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.468338    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f660a256-a12d-4a4a-823d-4b0ff5dcb3da-original-pull-secret\") pod \"global-pull-secret-syncer-rpdcd\" (UID: \"f660a256-a12d-4a4a-823d-4b0ff5dcb3da\") " pod="kube-system/global-pull-secret-syncer-rpdcd"
Oct 30 12:57:05.468398 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.468376    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/9a20c40d-f5f4-4d96-85b6-be69a6cd1828-multus-daemon-config\") pod \"multus-2l8x5\" (UID: \"9a20c40d-f5f4-4d96-85b6-be69a6cd1828\") " pod="openshift-multus/multus-2l8x5"
Oct 30 12:57:05.468558 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.468410    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/19b38c46-bfd4-4ed0-a764-f7b672f18cc6-telemetry-config\") pod \"cluster-monitoring-operator-884576c57-2sfj9\" (UID: \"19b38c46-bfd4-4ed0-a764-f7b672f18cc6\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-2sfj9"
Oct 30 12:57:05.468558 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.468461    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b11d3d97-ad88-4aa3-aaee-e83e3cf12f6d-metrics-certs\") pod \"network-metrics-daemon-p7tv6\" (UID: \"b11d3d97-ad88-4aa3-aaee-e83e3cf12f6d\") " pod="openshift-multus/network-metrics-daemon-p7tv6"
Oct 30 12:57:05.468558 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.468489    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/bd9e55af-64ad-4ffc-92c3-53b7bf654322-snapshots\") pod \"insights-operator-7c4986d449-7cfvt\" (UID: \"bd9e55af-64ad-4ffc-92c3-53b7bf654322\") " pod="openshift-insights/insights-operator-7c4986d449-7cfvt"
Oct 30 12:57:05.468558 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.468500    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qszn6\" (UniqueName: \"kubernetes.io/projected/e30bf7f2-4c3a-4801-8ef1-f0611812a9d4-kube-api-access-qszn6\") pod \"migrator-66df5d496b-cfhqz\" (UID: \"e30bf7f2-4c3a-4801-8ef1-f0611812a9d4\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-cfhqz"
Oct 30 12:57:05.468558 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.468537    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qzmnh\" (UniqueName: \"kubernetes.io/projected/9962cf52-bd50-4a97-a699-4da03085eb81-kube-api-access-qzmnh\") pod \"multus-additional-cni-plugins-m4p7x\" (UID: \"9962cf52-bd50-4a97-a699-4da03085eb81\") " pod="openshift-multus/multus-additional-cni-plugins-m4p7x"
Oct 30 12:57:05.468926 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.468689    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4817ec63-f4d5-4420-9417-148346ff8bb5-var-lib-openvswitch\") pod \"ovnkube-node-7rwcq\" (UID: \"4817ec63-f4d5-4420-9417-148346ff8bb5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7rwcq"
Oct 30 12:57:05.468926 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.468738    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/f660a256-a12d-4a4a-823d-4b0ff5dcb3da-dbus\") pod \"global-pull-secret-syncer-rpdcd\" (UID: \"f660a256-a12d-4a4a-823d-4b0ff5dcb3da\") " pod="kube-system/global-pull-secret-syncer-rpdcd"
Oct 30 12:57:05.468926 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.468879    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/ac1dc607-3072-4dbc-87e9-db316bf42dff-iptables-alerter-script\") pod \"iptables-alerter-kb9lc\" (UID: \"ac1dc607-3072-4dbc-87e9-db316bf42dff\") " pod="openshift-network-operator/iptables-alerter-kb9lc"
Oct 30 12:57:05.468926 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.468909    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/9a20c40d-f5f4-4d96-85b6-be69a6cd1828-multus-daemon-config\") pod \"multus-2l8x5\" (UID: \"9a20c40d-f5f4-4d96-85b6-be69a6cd1828\") " pod="openshift-multus/multus-2l8x5"
Oct 30 12:57:05.469109 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.468939    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8xmnf\" (UniqueName: \"kubernetes.io/projected/16402666-f516-49d1-be0d-f6ab7234e637-kube-api-access-8xmnf\") pod \"node-ca-h8q28\" (UID: \"16402666-f516-49d1-be0d-f6ab7234e637\") " pod="openshift-image-registry/node-ca-h8q28"
Oct 30 12:57:05.469109 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.469079    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9962cf52-bd50-4a97-a699-4da03085eb81-cnibin\") pod \"multus-additional-cni-plugins-m4p7x\" (UID: \"9962cf52-bd50-4a97-a699-4da03085eb81\") " pod="openshift-multus/multus-additional-cni-plugins-m4p7x"
Oct 30 12:57:05.469203 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.469132    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/56ff7ccd-573e-4bb2-a2b7-199a0346ed9f-tmp\") pod \"tuned-8mfkz\" (UID: \"56ff7ccd-573e-4bb2-a2b7-199a0346ed9f\") " pod="openshift-cluster-node-tuning-operator/tuned-8mfkz"
Oct 30 12:57:05.469250 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.469220    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9a20c40d-f5f4-4d96-85b6-be69a6cd1828-multus-conf-dir\") pod \"multus-2l8x5\" (UID: \"9a20c40d-f5f4-4d96-85b6-be69a6cd1828\") " pod="openshift-multus/multus-2l8x5"
Oct 30 12:57:05.469321 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.469275    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/55fa4842-3eb4-4e52-a4fc-18876ca409d5-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-knqz6\" (UID: \"55fa4842-3eb4-4e52-a4fc-18876ca409d5\") " pod="openshift-insights/insights-runtime-extractor-knqz6"
Oct 30 12:57:05.469380 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.469318    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/b86e8e45-3c2d-46b6-9ade-d390b56e4a75-installation-pull-secrets\") pod \"image-registry-69c87564f5-n6wjk\" (UID: \"b86e8e45-3c2d-46b6-9ade-d390b56e4a75\") " pod="openshift-image-registry/image-registry-69c87564f5-n6wjk"
Oct 30 12:57:05.469380 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.469219    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/ac1dc607-3072-4dbc-87e9-db316bf42dff-iptables-alerter-script\") pod \"iptables-alerter-kb9lc\" (UID: \"ac1dc607-3072-4dbc-87e9-db316bf42dff\") " pod="openshift-network-operator/iptables-alerter-kb9lc"
Oct 30 12:57:05.469473 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.469397    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/16402666-f516-49d1-be0d-f6ab7234e637-host\") pod \"node-ca-h8q28\" (UID: \"16402666-f516-49d1-be0d-f6ab7234e637\") " pod="openshift-image-registry/node-ca-h8q28"
Oct 30 12:57:05.469473 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.469436    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w2lch\" (UniqueName: \"kubernetes.io/projected/f07abb3f-8900-4fa5-8d35-c7ab1d35582d-kube-api-access-w2lch\") pod \"volume-data-source-validator-58c9f5f46d-xtfpg\" (UID: \"f07abb3f-8900-4fa5-8d35-c7ab1d35582d\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-xtfpg"
Oct 30 12:57:05.469570 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.469475    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4817ec63-f4d5-4420-9417-148346ff8bb5-etc-openvswitch\") pod \"ovnkube-node-7rwcq\" (UID: \"4817ec63-f4d5-4420-9417-148346ff8bb5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7rwcq"
Oct 30 12:57:05.469570 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.469516    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/56ff7ccd-573e-4bb2-a2b7-199a0346ed9f-tmp\") pod \"tuned-8mfkz\" (UID: \"56ff7ccd-573e-4bb2-a2b7-199a0346ed9f\") " pod="openshift-cluster-node-tuning-operator/tuned-8mfkz"
Oct 30 12:57:05.469570 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.469512    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/4817ec63-f4d5-4420-9417-148346ff8bb5-ovnkube-script-lib\") pod \"ovnkube-node-7rwcq\" (UID: \"4817ec63-f4d5-4420-9417-148346ff8bb5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7rwcq"
Oct 30 12:57:05.469803 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.469759    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/4883c26d-0877-4870-b0ee-406f5bfa8cf4-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-vsqgb\" (UID: \"4883c26d-0877-4870-b0ee-406f5bfa8cf4\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-vsqgb"
Oct 30 12:57:05.469803 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.469785    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/4817ec63-f4d5-4420-9417-148346ff8bb5-ovnkube-script-lib\") pod \"ovnkube-node-7rwcq\" (UID: \"4817ec63-f4d5-4420-9417-148346ff8bb5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7rwcq"
Oct 30 12:57:05.469803 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.469799    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rls7j\" (UniqueName: \"kubernetes.io/projected/85f4c399-0089-4399-86e2-069834f89286-kube-api-access-rls7j\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-plmkt\" (UID: \"85f4c399-0089-4399-86e2-069834f89286\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-plmkt"
Oct 30 12:57:05.469968 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.469894    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-config\" (UniqueName: \"kubernetes.io/configmap/23297e74-48f9-47b7-bb5a-80189bdbf2c4-samples-operator-config\") pod \"cluster-samples-operator-6ddb6c8b94-7dtcd\" (UID: \"23297e74-48f9-47b7-bb5a-80189bdbf2c4\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-7dtcd"
Oct 30 12:57:05.469968 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.469953    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/56ff7ccd-573e-4bb2-a2b7-199a0346ed9f-lib-modules\") pod \"tuned-8mfkz\" (UID: \"56ff7ccd-573e-4bb2-a2b7-199a0346ed9f\") " pod="openshift-cluster-node-tuning-operator/tuned-8mfkz"
Oct 30 12:57:05.470065 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.469998    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/56ff7ccd-573e-4bb2-a2b7-199a0346ed9f-var-lib-kubelet\") pod \"tuned-8mfkz\" (UID: \"56ff7ccd-573e-4bb2-a2b7-199a0346ed9f\") " pod="openshift-cluster-node-tuning-operator/tuned-8mfkz"
Oct 30 12:57:05.470230 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.470210    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"samples-operator-config\" (UniqueName: \"kubernetes.io/configmap/23297e74-48f9-47b7-bb5a-80189bdbf2c4-samples-operator-config\") pod \"cluster-samples-operator-6ddb6c8b94-7dtcd\" (UID: \"23297e74-48f9-47b7-bb5a-80189bdbf2c4\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-7dtcd"
Oct 30 12:57:05.470315 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.470286    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/eca286e8-0b02-42af-9d15-1d8670b7362c-tmp-dir\") pod \"node-resolver-4lfbk\" (UID: \"eca286e8-0b02-42af-9d15-1d8670b7362c\") " pod="openshift-dns/node-resolver-4lfbk"
Oct 30 12:57:05.470395 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.470320    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/98e7e1f3-2ca6-45de-a5d8-d27c22e257b1-cert\") pod \"ingress-canary-bdf5s\" (UID: \"98e7e1f3-2ca6-45de-a5d8-d27c22e257b1\") " pod="openshift-ingress-canary/ingress-canary-bdf5s"
Oct 30 12:57:05.470459 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.470401    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/1c405568-cb88-4485-ae2c-3614cc31ced6-installation-pull-secrets\") pod \"image-registry-77c6ff895d-vw6c6\" (UID: \"1c405568-cb88-4485-ae2c-3614cc31ced6\") " pod="openshift-image-registry/image-registry-77c6ff895d-vw6c6"
Oct 30 12:57:05.470459 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.470436    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/4817ec63-f4d5-4420-9417-148346ff8bb5-systemd-units\") pod \"ovnkube-node-7rwcq\" (UID: \"4817ec63-f4d5-4420-9417-148346ff8bb5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7rwcq"
Oct 30 12:57:05.470567 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.470467    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/56ff7ccd-573e-4bb2-a2b7-199a0346ed9f-etc-sysctl-conf\") pod \"tuned-8mfkz\" (UID: \"56ff7ccd-573e-4bb2-a2b7-199a0346ed9f\") " pod="openshift-cluster-node-tuning-operator/tuned-8mfkz"
Oct 30 12:57:05.470567 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.470499    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/9a20c40d-f5f4-4d96-85b6-be69a6cd1828-multus-socket-dir-parent\") pod \"multus-2l8x5\" (UID: \"9a20c40d-f5f4-4d96-85b6-be69a6cd1828\") " pod="openshift-multus/multus-2l8x5"
Oct 30 12:57:05.470567 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.470561    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9a20c40d-f5f4-4d96-85b6-be69a6cd1828-host-var-lib-cni-bin\") pod \"multus-2l8x5\" (UID: \"9a20c40d-f5f4-4d96-85b6-be69a6cd1828\") " pod="openshift-multus/multus-2l8x5"
Oct 30 12:57:05.470722 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.470596    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/16402666-f516-49d1-be0d-f6ab7234e637-serviceca\") pod \"node-ca-h8q28\" (UID: \"16402666-f516-49d1-be0d-f6ab7234e637\") " pod="openshift-image-registry/node-ca-h8q28"
Oct 30 12:57:05.470722 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.470628    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/51d01732-87a4-42b0-a6d5-a3fac689d7f6-konnectivity-ca\") pod \"konnectivity-agent-zth9k\" (UID: \"51d01732-87a4-42b0-a6d5-a3fac689d7f6\") " pod="kube-system/konnectivity-agent-zth9k"
Oct 30 12:57:05.470722 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.470654    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4817ec63-f4d5-4420-9417-148346ff8bb5-host-cni-bin\") pod \"ovnkube-node-7rwcq\" (UID: \"4817ec63-f4d5-4420-9417-148346ff8bb5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7rwcq"
Oct 30 12:57:05.470722 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.470687    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/56ff7ccd-573e-4bb2-a2b7-199a0346ed9f-etc-sysctl-d\") pod \"tuned-8mfkz\" (UID: \"56ff7ccd-573e-4bb2-a2b7-199a0346ed9f\") " pod="openshift-cluster-node-tuning-operator/tuned-8mfkz"
Oct 30 12:57:05.470722 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.470718    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/9a20c40d-f5f4-4d96-85b6-be69a6cd1828-host-run-k8s-cni-cncf-io\") pod \"multus-2l8x5\" (UID: \"9a20c40d-f5f4-4d96-85b6-be69a6cd1828\") " pod="openshift-multus/multus-2l8x5"
Oct 30 12:57:05.471004 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.470752    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c6k8q\" (UniqueName: \"kubernetes.io/projected/9a20c40d-f5f4-4d96-85b6-be69a6cd1828-kube-api-access-c6k8q\") pod \"multus-2l8x5\" (UID: \"9a20c40d-f5f4-4d96-85b6-be69a6cd1828\") " pod="openshift-multus/multus-2l8x5"
Oct 30 12:57:05.471004 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.470777    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/fcf99170-f5cf-4987-97bd-1c30b1b37a35-metrics-tls\") pod \"dns-default-g2ckh\" (UID: \"fcf99170-f5cf-4987-97bd-1c30b1b37a35\") " pod="openshift-dns/dns-default-g2ckh"
Oct 30 12:57:05.471004 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.470885    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fcf99170-f5cf-4987-97bd-1c30b1b37a35-tmp-dir\") pod \"dns-default-g2ckh\" (UID: \"fcf99170-f5cf-4987-97bd-1c30b1b37a35\") " pod="openshift-dns/dns-default-g2ckh"
Oct 30 12:57:05.471004 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.470930    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gz9d6\" (UniqueName: \"kubernetes.io/projected/7ed65395-2cbd-46bf-81dc-fe307f3a9345-kube-api-access-gz9d6\") pod \"network-check-source-6f78d59f99-vdwgx\" (UID: \"7ed65395-2cbd-46bf-81dc-fe307f3a9345\") " pod="openshift-network-diagnostics/network-check-source-6f78d59f99-vdwgx"
Oct 30 12:57:05.471004 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.470965    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/a3ceb095-c75c-43b3-9de5-7dda644eddb1-device-dir\") pod \"aws-ebs-csi-driver-node-8bnjr\" (UID: \"a3ceb095-c75c-43b3-9de5-7dda644eddb1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8bnjr"
Oct 30 12:57:05.471004 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.470990    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9a20c40d-f5f4-4d96-85b6-be69a6cd1828-cni-binary-copy\") pod \"multus-2l8x5\" (UID: \"9a20c40d-f5f4-4d96-85b6-be69a6cd1828\") " pod="openshift-multus/multus-2l8x5"
Oct 30 12:57:05.471306 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.471022    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vtsfh\" (UniqueName: \"kubernetes.io/projected/b11d3d97-ad88-4aa3-aaee-e83e3cf12f6d-kube-api-access-vtsfh\") pod \"network-metrics-daemon-p7tv6\" (UID: \"b11d3d97-ad88-4aa3-aaee-e83e3cf12f6d\") " pod="openshift-multus/network-metrics-daemon-p7tv6"
Oct 30 12:57:05.471306 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.471053    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/1c405568-cb88-4485-ae2c-3614cc31ced6-trusted-ca\") pod \"image-registry-77c6ff895d-vw6c6\" (UID: \"1c405568-cb88-4485-ae2c-3614cc31ced6\") " pod="openshift-image-registry/image-registry-77c6ff895d-vw6c6"
Oct 30 12:57:05.471306 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.471083    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/b86e8e45-3c2d-46b6-9ade-d390b56e4a75-registry-certificates\") pod \"image-registry-69c87564f5-n6wjk\" (UID: \"b86e8e45-3c2d-46b6-9ade-d390b56e4a75\") " pod="openshift-image-registry/image-registry-69c87564f5-n6wjk"
Oct 30 12:57:05.471306 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.471116    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/a3ceb095-c75c-43b3-9de5-7dda644eddb1-etc-selinux\") pod \"aws-ebs-csi-driver-node-8bnjr\" (UID: \"a3ceb095-c75c-43b3-9de5-7dda644eddb1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8bnjr"
Oct 30 12:57:05.471306 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.471142    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/56ff7ccd-573e-4bb2-a2b7-199a0346ed9f-etc-kubernetes\") pod \"tuned-8mfkz\" (UID: \"56ff7ccd-573e-4bb2-a2b7-199a0346ed9f\") " pod="openshift-cluster-node-tuning-operator/tuned-8mfkz"
Oct 30 12:57:05.471306 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.471173    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/9a20c40d-f5f4-4d96-85b6-be69a6cd1828-host-var-lib-cni-multus\") pod \"multus-2l8x5\" (UID: \"9a20c40d-f5f4-4d96-85b6-be69a6cd1828\") " pod="openshift-multus/multus-2l8x5"
Oct 30 12:57:05.471306 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.471204    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/bd9e55af-64ad-4ffc-92c3-53b7bf654322-serving-cert\") pod \"insights-operator-7c4986d449-7cfvt\" (UID: \"bd9e55af-64ad-4ffc-92c3-53b7bf654322\") " pod="openshift-insights/insights-operator-7c4986d449-7cfvt"
Oct 30 12:57:05.471306 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.471233    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/56ff7ccd-573e-4bb2-a2b7-199a0346ed9f-sys\") pod \"tuned-8mfkz\" (UID: \"56ff7ccd-573e-4bb2-a2b7-199a0346ed9f\") " pod="openshift-cluster-node-tuning-operator/tuned-8mfkz"
Oct 30 12:57:05.471306 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.471285    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-66dlr\" (UniqueName: \"kubernetes.io/projected/56ff7ccd-573e-4bb2-a2b7-199a0346ed9f-kube-api-access-66dlr\") pod \"tuned-8mfkz\" (UID: \"56ff7ccd-573e-4bb2-a2b7-199a0346ed9f\") " pod="openshift-cluster-node-tuning-operator/tuned-8mfkz"
Oct 30 12:57:05.471700 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.471320    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b86e8e45-3c2d-46b6-9ade-d390b56e4a75-registry-tls\") pod \"image-registry-69c87564f5-n6wjk\" (UID: \"b86e8e45-3c2d-46b6-9ade-d390b56e4a75\") " pod="openshift-image-registry/image-registry-69c87564f5-n6wjk"
Oct 30 12:57:05.471700 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.471353    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jz42t\" (UniqueName: \"kubernetes.io/projected/23297e74-48f9-47b7-bb5a-80189bdbf2c4-kube-api-access-jz42t\") pod \"cluster-samples-operator-6ddb6c8b94-7dtcd\" (UID: \"23297e74-48f9-47b7-bb5a-80189bdbf2c4\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-7dtcd"
Oct 30 12:57:05.471700 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.471554    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/4817ec63-f4d5-4420-9417-148346ff8bb5-host-slash\") pod \"ovnkube-node-7rwcq\" (UID: \"4817ec63-f4d5-4420-9417-148346ff8bb5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7rwcq"
Oct 30 12:57:05.471700 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.471586    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4817ec63-f4d5-4420-9417-148346ff8bb5-host-run-netns\") pod \"ovnkube-node-7rwcq\" (UID: \"4817ec63-f4d5-4420-9417-148346ff8bb5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7rwcq"
Oct 30 12:57:05.471700 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.471618    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/4817ec63-f4d5-4420-9417-148346ff8bb5-env-overrides\") pod \"ovnkube-node-7rwcq\" (UID: \"4817ec63-f4d5-4420-9417-148346ff8bb5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7rwcq"
Oct 30 12:57:05.471700 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.471647    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/4817ec63-f4d5-4420-9417-148346ff8bb5-ovn-node-metrics-cert\") pod \"ovnkube-node-7rwcq\" (UID: \"4817ec63-f4d5-4420-9417-148346ff8bb5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7rwcq"
Oct 30 12:57:05.471700 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.471679    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hbsbr\" (UniqueName: \"kubernetes.io/projected/98e7e1f3-2ca6-45de-a5d8-d27c22e257b1-kube-api-access-hbsbr\") pod \"ingress-canary-bdf5s\" (UID: \"98e7e1f3-2ca6-45de-a5d8-d27c22e257b1\") " pod="openshift-ingress-canary/ingress-canary-bdf5s"
Oct 30 12:57:05.472071 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.471709    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1c405568-cb88-4485-ae2c-3614cc31ced6-registry-tls\") pod \"image-registry-77c6ff895d-vw6c6\" (UID: \"1c405568-cb88-4485-ae2c-3614cc31ced6\") " pod="openshift-image-registry/image-registry-77c6ff895d-vw6c6"
Oct 30 12:57:05.472071 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.471741    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/b86e8e45-3c2d-46b6-9ade-d390b56e4a75-image-registry-private-configuration\") pod \"image-registry-69c87564f5-n6wjk\" (UID: \"b86e8e45-3c2d-46b6-9ade-d390b56e4a75\") " pod="openshift-image-registry/image-registry-69c87564f5-n6wjk"
Oct 30 12:57:05.472071 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.471782    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6sjmw\" (UniqueName: \"kubernetes.io/projected/a4d58318-42ba-4a9e-9c74-43fe90baa4a7-kube-api-access-6sjmw\") pod \"kas-connection-checker-685895c7c9-4zl6m\" (UID: \"a4d58318-42ba-4a9e-9c74-43fe90baa4a7\") " pod="kube-system/kas-connection-checker-685895c7c9-4zl6m"
Oct 30 12:57:05.472071 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.471813    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qt8c9\" (UniqueName: \"kubernetes.io/projected/bd9e55af-64ad-4ffc-92c3-53b7bf654322-kube-api-access-qt8c9\") pod \"insights-operator-7c4986d449-7cfvt\" (UID: \"bd9e55af-64ad-4ffc-92c3-53b7bf654322\") " pod="openshift-insights/insights-operator-7c4986d449-7cfvt"
Oct 30 12:57:05.472071 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.471844    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tltj6\" (UniqueName: \"kubernetes.io/projected/d8930ee7-0ae0-48b5-8ce3-cb8820355cef-kube-api-access-tltj6\") pod \"console-operator-7cd56b8dbc-8fqfr\" (UID: \"d8930ee7-0ae0-48b5-8ce3-cb8820355cef\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-8fqfr"
Oct 30 12:57:05.472071 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.471875    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/b86e8e45-3c2d-46b6-9ade-d390b56e4a75-ca-trust-extracted\") pod \"image-registry-69c87564f5-n6wjk\" (UID: \"b86e8e45-3c2d-46b6-9ade-d390b56e4a75\") " pod="openshift-image-registry/image-registry-69c87564f5-n6wjk"
Oct 30 12:57:05.472071 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.471901    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/4817ec63-f4d5-4420-9417-148346ff8bb5-run-ovn\") pod \"ovnkube-node-7rwcq\" (UID: \"4817ec63-f4d5-4420-9417-148346ff8bb5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7rwcq"
Oct 30 12:57:05.472422 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.472049    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fcf99170-f5cf-4987-97bd-1c30b1b37a35-tmp-dir\") pod \"dns-default-g2ckh\" (UID: \"fcf99170-f5cf-4987-97bd-1c30b1b37a35\") " pod="openshift-dns/dns-default-g2ckh"
Oct 30 12:57:05.472422 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.472122    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/b86e8e45-3c2d-46b6-9ade-d390b56e4a75-ca-trust-extracted\") pod \"image-registry-69c87564f5-n6wjk\" (UID: \"b86e8e45-3c2d-46b6-9ade-d390b56e4a75\") " pod="openshift-image-registry/image-registry-69c87564f5-n6wjk"
Oct 30 12:57:05.473008 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.472988    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/4817ec63-f4d5-4420-9417-148346ff8bb5-env-overrides\") pod \"ovnkube-node-7rwcq\" (UID: \"4817ec63-f4d5-4420-9417-148346ff8bb5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7rwcq"
Oct 30 12:57:05.473080 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.473027    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/b86e8e45-3c2d-46b6-9ade-d390b56e4a75-registry-certificates\") pod \"image-registry-69c87564f5-n6wjk\" (UID: \"b86e8e45-3c2d-46b6-9ade-d390b56e4a75\") " pod="openshift-image-registry/image-registry-69c87564f5-n6wjk"
Oct 30 12:57:05.473132 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.473081    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/eca286e8-0b02-42af-9d15-1d8670b7362c-tmp-dir\") pod \"node-resolver-4lfbk\" (UID: \"eca286e8-0b02-42af-9d15-1d8670b7362c\") " pod="openshift-dns/node-resolver-4lfbk"
Oct 30 12:57:05.473132 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.473108    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/1c405568-cb88-4485-ae2c-3614cc31ced6-trusted-ca\") pod \"image-registry-77c6ff895d-vw6c6\" (UID: \"1c405568-cb88-4485-ae2c-3614cc31ced6\") " pod="openshift-image-registry/image-registry-77c6ff895d-vw6c6"
Oct 30 12:57:05.473750 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.473731    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/16402666-f516-49d1-be0d-f6ab7234e637-serviceca\") pod \"node-ca-h8q28\" (UID: \"16402666-f516-49d1-be0d-f6ab7234e637\") " pod="openshift-image-registry/node-ca-h8q28"
Oct 30 12:57:05.473820 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.473764    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/51d01732-87a4-42b0-a6d5-a3fac689d7f6-konnectivity-ca\") pod \"konnectivity-agent-zth9k\" (UID: \"51d01732-87a4-42b0-a6d5-a3fac689d7f6\") " pod="kube-system/konnectivity-agent-zth9k"
Oct 30 12:57:05.473876 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.473815    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/19b38c46-bfd4-4ed0-a764-f7b672f18cc6-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-2sfj9\" (UID: \"19b38c46-bfd4-4ed0-a764-f7b672f18cc6\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-2sfj9"
Oct 30 12:57:05.473876 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.473831    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9a20c40d-f5f4-4d96-85b6-be69a6cd1828-cni-binary-copy\") pod \"multus-2l8x5\" (UID: \"9a20c40d-f5f4-4d96-85b6-be69a6cd1828\") " pod="openshift-multus/multus-2l8x5"
Oct 30 12:57:05.473876 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.473846    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9r47j\" (UniqueName: \"kubernetes.io/projected/ac1dc607-3072-4dbc-87e9-db316bf42dff-kube-api-access-9r47j\") pod \"iptables-alerter-kb9lc\" (UID: \"ac1dc607-3072-4dbc-87e9-db316bf42dff\") " pod="openshift-network-operator/iptables-alerter-kb9lc"
Oct 30 12:57:05.474037 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.473877    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/fcf99170-f5cf-4987-97bd-1c30b1b37a35-config-volume\") pod \"dns-default-g2ckh\" (UID: \"fcf99170-f5cf-4987-97bd-1c30b1b37a35\") " pod="openshift-dns/dns-default-g2ckh"
Oct 30 12:57:05.474037 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.473950    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zg85t\" (UniqueName: \"kubernetes.io/projected/55fa4842-3eb4-4e52-a4fc-18876ca409d5-kube-api-access-zg85t\") pod \"insights-runtime-extractor-knqz6\" (UID: \"55fa4842-3eb4-4e52-a4fc-18876ca409d5\") " pod="openshift-insights/insights-runtime-extractor-knqz6"
Oct 30 12:57:05.474037 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.473979    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/a3ceb095-c75c-43b3-9de5-7dda644eddb1-registration-dir\") pod \"aws-ebs-csi-driver-node-8bnjr\" (UID: \"a3ceb095-c75c-43b3-9de5-7dda644eddb1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8bnjr"
Oct 30 12:57:05.474037 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.473996    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/a3ceb095-c75c-43b3-9de5-7dda644eddb1-sys-fs\") pod \"aws-ebs-csi-driver-node-8bnjr\" (UID: \"a3ceb095-c75c-43b3-9de5-7dda644eddb1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8bnjr"
Oct 30 12:57:05.474037 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.474015    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6rhvm\" (UniqueName: \"kubernetes.io/projected/a3ceb095-c75c-43b3-9de5-7dda644eddb1-kube-api-access-6rhvm\") pod \"aws-ebs-csi-driver-node-8bnjr\" (UID: \"a3ceb095-c75c-43b3-9de5-7dda644eddb1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8bnjr"
Oct 30 12:57:05.474272 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.474044    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/fcf99170-f5cf-4987-97bd-1c30b1b37a35-config-volume\") pod \"dns-default-g2ckh\" (UID: \"fcf99170-f5cf-4987-97bd-1c30b1b37a35\") " pod="openshift-dns/dns-default-g2ckh"
Oct 30 12:57:05.474272 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.474079    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/b86e8e45-3c2d-46b6-9ade-d390b56e4a75-image-registry-private-configuration\") pod \"image-registry-69c87564f5-n6wjk\" (UID: \"b86e8e45-3c2d-46b6-9ade-d390b56e4a75\") " pod="openshift-image-registry/image-registry-69c87564f5-n6wjk"
Oct 30 12:57:05.474272 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.474144    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/56ff7ccd-573e-4bb2-a2b7-199a0346ed9f-etc-sysconfig\") pod \"tuned-8mfkz\" (UID: \"56ff7ccd-573e-4bb2-a2b7-199a0346ed9f\") " pod="openshift-cluster-node-tuning-operator/tuned-8mfkz"
Oct 30 12:57:05.474272 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.474200    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/9a20c40d-f5f4-4d96-85b6-be69a6cd1828-hostroot\") pod \"multus-2l8x5\" (UID: \"9a20c40d-f5f4-4d96-85b6-be69a6cd1828\") " pod="openshift-multus/multus-2l8x5"
Oct 30 12:57:05.474272 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.474223    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/4817ec63-f4d5-4420-9417-148346ff8bb5-node-log\") pod \"ovnkube-node-7rwcq\" (UID: \"4817ec63-f4d5-4420-9417-148346ff8bb5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7rwcq"
Oct 30 12:57:05.474272 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.474244    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/56ff7ccd-573e-4bb2-a2b7-199a0346ed9f-etc-tuned\") pod \"tuned-8mfkz\" (UID: \"56ff7ccd-573e-4bb2-a2b7-199a0346ed9f\") " pod="openshift-cluster-node-tuning-operator/tuned-8mfkz"
Oct 30 12:57:05.474602 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.474298    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/eca286e8-0b02-42af-9d15-1d8670b7362c-hosts-file\") pod \"node-resolver-4lfbk\" (UID: \"eca286e8-0b02-42af-9d15-1d8670b7362c\") " pod="openshift-dns/node-resolver-4lfbk"
Oct 30 12:57:05.474602 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.474310    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/56ff7ccd-573e-4bb2-a2b7-199a0346ed9f-etc-tuned\") pod \"tuned-8mfkz\" (UID: \"56ff7ccd-573e-4bb2-a2b7-199a0346ed9f\") " pod="openshift-cluster-node-tuning-operator/tuned-8mfkz"
Oct 30 12:57:05.474602 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.474348    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/1c405568-cb88-4485-ae2c-3614cc31ced6-ca-trust-extracted\") pod \"image-registry-77c6ff895d-vw6c6\" (UID: \"1c405568-cb88-4485-ae2c-3614cc31ced6\") " pod="openshift-image-registry/image-registry-77c6ff895d-vw6c6"
Oct 30 12:57:05.474602 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.474398    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/a3ceb095-c75c-43b3-9de5-7dda644eddb1-socket-dir\") pod \"aws-ebs-csi-driver-node-8bnjr\" (UID: \"a3ceb095-c75c-43b3-9de5-7dda644eddb1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8bnjr"
Oct 30 12:57:05.474602 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.474402    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/1c405568-cb88-4485-ae2c-3614cc31ced6-ca-trust-extracted\") pod \"image-registry-77c6ff895d-vw6c6\" (UID: \"1c405568-cb88-4485-ae2c-3614cc31ced6\") " pod="openshift-image-registry/image-registry-77c6ff895d-vw6c6"
Oct 30 12:57:05.474602 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.474426    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/4817ec63-f4d5-4420-9417-148346ff8bb5-host-kubelet\") pod \"ovnkube-node-7rwcq\" (UID: \"4817ec63-f4d5-4420-9417-148346ff8bb5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7rwcq"
Oct 30 12:57:05.474602 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.474483    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4817ec63-f4d5-4420-9417-148346ff8bb5-run-openvswitch\") pod \"ovnkube-node-7rwcq\" (UID: \"4817ec63-f4d5-4420-9417-148346ff8bb5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7rwcq"
Oct 30 12:57:05.474602 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.474509    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9a20c40d-f5f4-4d96-85b6-be69a6cd1828-multus-cni-dir\") pod \"multus-2l8x5\" (UID: \"9a20c40d-f5f4-4d96-85b6-be69a6cd1828\") " pod="openshift-multus/multus-2l8x5"
Oct 30 12:57:05.490021 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.490005    4943 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Oct 30 12:57:05.490256 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.490241    4943 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\""
Oct 30 12:57:05.490334 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.490273    4943 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\""
Oct 30 12:57:05.490334 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.490299    4943 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
Oct 30 12:57:05.490470 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.490379    4943 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\""
Oct 30 12:57:05.490470 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.490381    4943 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Oct 30 12:57:05.490691 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.490675    4943 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-5bbfv\""
Oct 30 12:57:05.490747 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.490702    4943 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Oct 30 12:57:05.490747 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.490713    4943 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Oct 30 12:57:05.497000 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.496986    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/85f4c399-0089-4399-86e2-069834f89286-serving-cert\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-plmkt\" (UID: \"85f4c399-0089-4399-86e2-069834f89286\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-plmkt"
Oct 30 12:57:05.497078 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.497064    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/27c55f84-4e9a-44de-94ea-02258d6bc747-serving-cert\") pod \"service-ca-operator-544d749c8c-h42v2\" (UID: \"27c55f84-4e9a-44de-94ea-02258d6bc747\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-h42v2"
Oct 30 12:57:05.497890 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.497877    4943 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Oct 30 12:57:05.497981 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.497968    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/e1f629e6-6a96-444f-8329-e71bb98c9029-default-certificate\") pod \"router-default-649bdc978f-lkpqz\" (UID: \"e1f629e6-6a96-444f-8329-e71bb98c9029\") " pod="openshift-ingress/router-default-649bdc978f-lkpqz"
Oct 30 12:57:05.498078 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.498059    4943 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-94phb\""
Oct 30 12:57:05.498078 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.498056    4943 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-wf8rx\""
Oct 30 12:57:05.498204 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.498125    4943 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-node-identity-dockercfg-sp79l\""
Oct 30 12:57:05.498204 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.498182    4943 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\""
Oct 30 12:57:05.498289 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.498233    4943 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-7gdbz\""
Oct 30 12:57:05.498419 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.498402    4943 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Oct 30 12:57:05.498693 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.498487    4943 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-cxt4s\""
Oct 30 12:57:05.498693 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.498488    4943 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-kmvh7\""
Oct 30 12:57:05.498693 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.498517    4943 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-hp484\""
Oct 30 12:57:05.498693 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.498563    4943 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-v4tv9\""
Oct 30 12:57:05.498693 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.498583    4943 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-dockercfg-7lnpl\""
Oct 30 12:57:05.498693 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.498629    4943 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"ingress-canary-dockercfg-sdfnl\""
Oct 30 12:57:05.498693 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.498633    4943 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Oct 30 12:57:05.498693 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.498644    4943 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent-dockercfg-wsrzk\""
Oct 30 12:57:05.498693 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.498631    4943 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Oct 30 12:57:05.498693 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.498665    4943 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-tzbll\""
Oct 30 12:57:05.499188 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.498707    4943 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-xt92t\""
Oct 30 12:57:05.499188 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.499154    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rls7j\" (UniqueName: \"kubernetes.io/projected/85f4c399-0089-4399-86e2-069834f89286-kube-api-access-rls7j\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-plmkt\" (UID: \"85f4c399-0089-4399-86e2-069834f89286\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-plmkt"
Oct 30 12:57:05.499366 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.499214    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qzmnh\" (UniqueName: \"kubernetes.io/projected/9962cf52-bd50-4a97-a699-4da03085eb81-kube-api-access-qzmnh\") pod \"multus-additional-cni-plugins-m4p7x\" (UID: \"9962cf52-bd50-4a97-a699-4da03085eb81\") " pod="openshift-multus/multus-additional-cni-plugins-m4p7x"
Oct 30 12:57:05.499366 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.499224    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qxmjj\" (UniqueName: \"kubernetes.io/projected/fcf99170-f5cf-4987-97bd-1c30b1b37a35-kube-api-access-qxmjj\") pod \"dns-default-g2ckh\" (UID: \"fcf99170-f5cf-4987-97bd-1c30b1b37a35\") " pod="openshift-dns/dns-default-g2ckh"
Oct 30 12:57:05.499366 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.499287    4943 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Oct 30 12:57:05.499366 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.499297    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7rwfx\" (UniqueName: \"kubernetes.io/projected/4817ec63-f4d5-4420-9417-148346ff8bb5-kube-api-access-7rwfx\") pod \"ovnkube-node-7rwcq\" (UID: \"4817ec63-f4d5-4420-9417-148346ff8bb5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7rwcq"
Oct 30 12:57:05.499366 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.499317    4943 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-25fk6\""
Oct 30 12:57:05.499366 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.499317    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-49z5c\" (UniqueName: \"kubernetes.io/projected/301eebcb-3d48-4eef-a0de-a1173e23ad65-kube-api-access-49z5c\") pod \"kas-connection-checker-685895c7c9-h7brb\" (UID: \"301eebcb-3d48-4eef-a0de-a1173e23ad65\") " pod="kube-system/kas-connection-checker-685895c7c9-h7brb"
Oct 30 12:57:05.499366 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.499349    4943 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-sqtw6\""
Oct 30 12:57:05.499366 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.499350    4943 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-dz42j\""
Oct 30 12:57:05.499366 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.499359    4943 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-vg7bt\""
Oct 30 12:57:05.499623 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.499462    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-54kft\" (UniqueName: \"kubernetes.io/projected/1c405568-cb88-4485-ae2c-3614cc31ced6-kube-api-access-54kft\") pod \"image-registry-77c6ff895d-vw6c6\" (UID: \"1c405568-cb88-4485-ae2c-3614cc31ced6\") " pod="openshift-image-registry/image-registry-77c6ff895d-vw6c6"
Oct 30 12:57:05.499623 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.499502    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l8pqr\" (UniqueName: \"kubernetes.io/projected/b86e8e45-3c2d-46b6-9ade-d390b56e4a75-kube-api-access-l8pqr\") pod \"image-registry-69c87564f5-n6wjk\" (UID: \"b86e8e45-3c2d-46b6-9ade-d390b56e4a75\") " pod="openshift-image-registry/image-registry-69c87564f5-n6wjk"
Oct 30 12:57:05.500169 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.500152    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/b86e8e45-3c2d-46b6-9ade-d390b56e4a75-installation-pull-secrets\") pod \"image-registry-69c87564f5-n6wjk\" (UID: \"b86e8e45-3c2d-46b6-9ade-d390b56e4a75\") " pod="openshift-image-registry/image-registry-69c87564f5-n6wjk"
Oct 30 12:57:05.500493 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.500477    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qszn6\" (UniqueName: \"kubernetes.io/projected/e30bf7f2-4c3a-4801-8ef1-f0611812a9d4-kube-api-access-qszn6\") pod \"migrator-66df5d496b-cfhqz\" (UID: \"e30bf7f2-4c3a-4801-8ef1-f0611812a9d4\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-cfhqz"
Oct 30 12:57:05.501249 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.501234    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1c405568-cb88-4485-ae2c-3614cc31ced6-bound-sa-token\") pod \"image-registry-77c6ff895d-vw6c6\" (UID: \"1c405568-cb88-4485-ae2c-3614cc31ced6\") " pod="openshift-image-registry/image-registry-77c6ff895d-vw6c6"
Oct 30 12:57:05.502592 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.502577    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/bd9e55af-64ad-4ffc-92c3-53b7bf654322-serving-cert\") pod \"insights-operator-7c4986d449-7cfvt\" (UID: \"bd9e55af-64ad-4ffc-92c3-53b7bf654322\") " pod="openshift-insights/insights-operator-7c4986d449-7cfvt"
Oct 30 12:57:05.502776 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.502761    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hfnqv\" (UniqueName: \"kubernetes.io/projected/27c55f84-4e9a-44de-94ea-02258d6bc747-kube-api-access-hfnqv\") pod \"service-ca-operator-544d749c8c-h42v2\" (UID: \"27c55f84-4e9a-44de-94ea-02258d6bc747\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-h42v2"
Oct 30 12:57:05.503621 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.503601    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/1c405568-cb88-4485-ae2c-3614cc31ced6-installation-pull-secrets\") pod \"image-registry-77c6ff895d-vw6c6\" (UID: \"1c405568-cb88-4485-ae2c-3614cc31ced6\") " pod="openshift-image-registry/image-registry-77c6ff895d-vw6c6"
Oct 30 12:57:05.503696 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.503646    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/4817ec63-f4d5-4420-9417-148346ff8bb5-ovn-node-metrics-cert\") pod \"ovnkube-node-7rwcq\" (UID: \"4817ec63-f4d5-4420-9417-148346ff8bb5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7rwcq"
Oct 30 12:57:05.503696 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.503682    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/98e7e1f3-2ca6-45de-a5d8-d27c22e257b1-cert\") pod \"ingress-canary-bdf5s\" (UID: \"98e7e1f3-2ca6-45de-a5d8-d27c22e257b1\") " pod="openshift-ingress-canary/ingress-canary-bdf5s"
Oct 30 12:57:05.504991 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.504975    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-swxqr\" (UniqueName: \"kubernetes.io/projected/0905782c-26ad-4dc8-93cf-d6f211ea7af5-kube-api-access-swxqr\") pod \"kas-connection-checker-685895c7c9-k4v7j\" (UID: \"0905782c-26ad-4dc8-93cf-d6f211ea7af5\") " pod="kube-system/kas-connection-checker-685895c7c9-k4v7j"
Oct 30 12:57:05.505786 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.505770    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b86e8e45-3c2d-46b6-9ade-d390b56e4a75-bound-sa-token\") pod \"image-registry-69c87564f5-n6wjk\" (UID: \"b86e8e45-3c2d-46b6-9ade-d390b56e4a75\") " pod="openshift-image-registry/image-registry-69c87564f5-n6wjk"
Oct 30 12:57:05.505900 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.505884    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j2mgz\" (UniqueName: \"kubernetes.io/projected/eca286e8-0b02-42af-9d15-1d8670b7362c-kube-api-access-j2mgz\") pod \"node-resolver-4lfbk\" (UID: \"eca286e8-0b02-42af-9d15-1d8670b7362c\") " pod="openshift-dns/node-resolver-4lfbk"
Oct 30 12:57:05.506039 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.506024    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8xmnf\" (UniqueName: \"kubernetes.io/projected/16402666-f516-49d1-be0d-f6ab7234e637-kube-api-access-8xmnf\") pod \"node-ca-h8q28\" (UID: \"16402666-f516-49d1-be0d-f6ab7234e637\") " pod="openshift-image-registry/node-ca-h8q28"
Oct 30 12:57:05.506600 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.506584    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6sjmw\" (UniqueName: \"kubernetes.io/projected/a4d58318-42ba-4a9e-9c74-43fe90baa4a7-kube-api-access-6sjmw\") pod \"kas-connection-checker-685895c7c9-4zl6m\" (UID: \"a4d58318-42ba-4a9e-9c74-43fe90baa4a7\") " pod="kube-system/kas-connection-checker-685895c7c9-4zl6m"
Oct 30 12:57:05.506903 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.506889    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qt8c9\" (UniqueName: \"kubernetes.io/projected/bd9e55af-64ad-4ffc-92c3-53b7bf654322-kube-api-access-qt8c9\") pod \"insights-operator-7c4986d449-7cfvt\" (UID: \"bd9e55af-64ad-4ffc-92c3-53b7bf654322\") " pod="openshift-insights/insights-operator-7c4986d449-7cfvt"
Oct 30 12:57:05.507007 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.506993    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tltj6\" (UniqueName: \"kubernetes.io/projected/d8930ee7-0ae0-48b5-8ce3-cb8820355cef-kube-api-access-tltj6\") pod \"console-operator-7cd56b8dbc-8fqfr\" (UID: \"d8930ee7-0ae0-48b5-8ce3-cb8820355cef\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-8fqfr"
Oct 30 12:57:05.508196 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.508182    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/51d01732-87a4-42b0-a6d5-a3fac689d7f6-agent-certs\") pod \"konnectivity-agent-zth9k\" (UID: \"51d01732-87a4-42b0-a6d5-a3fac689d7f6\") " pod="kube-system/konnectivity-agent-zth9k"
Oct 30 12:57:05.508620 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.508606    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9r47j\" (UniqueName: \"kubernetes.io/projected/ac1dc607-3072-4dbc-87e9-db316bf42dff-kube-api-access-9r47j\") pod \"iptables-alerter-kb9lc\" (UID: \"ac1dc607-3072-4dbc-87e9-db316bf42dff\") " pod="openshift-network-operator/iptables-alerter-kb9lc"
Oct 30 12:57:05.508784 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.508769    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qwmfv\" (UniqueName: \"kubernetes.io/projected/19b38c46-bfd4-4ed0-a764-f7b672f18cc6-kube-api-access-qwmfv\") pod \"cluster-monitoring-operator-884576c57-2sfj9\" (UID: \"19b38c46-bfd4-4ed0-a764-f7b672f18cc6\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-2sfj9"
Oct 30 12:57:05.508933 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.508920    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f660a256-a12d-4a4a-823d-4b0ff5dcb3da-original-pull-secret\") pod \"global-pull-secret-syncer-rpdcd\" (UID: \"f660a256-a12d-4a4a-823d-4b0ff5dcb3da\") " pod="kube-system/global-pull-secret-syncer-rpdcd"
Oct 30 12:57:05.509548 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.509533    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w2lch\" (UniqueName: \"kubernetes.io/projected/f07abb3f-8900-4fa5-8d35-c7ab1d35582d-kube-api-access-w2lch\") pod \"volume-data-source-validator-58c9f5f46d-xtfpg\" (UID: \"f07abb3f-8900-4fa5-8d35-c7ab1d35582d\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-xtfpg"
Oct 30 12:57:05.510040 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.510024    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jz42t\" (UniqueName: \"kubernetes.io/projected/23297e74-48f9-47b7-bb5a-80189bdbf2c4-kube-api-access-jz42t\") pod \"cluster-samples-operator-6ddb6c8b94-7dtcd\" (UID: \"23297e74-48f9-47b7-bb5a-80189bdbf2c4\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-7dtcd"
Oct 30 12:57:05.510119 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.510104    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c6k8q\" (UniqueName: \"kubernetes.io/projected/9a20c40d-f5f4-4d96-85b6-be69a6cd1828-kube-api-access-c6k8q\") pod \"multus-2l8x5\" (UID: \"9a20c40d-f5f4-4d96-85b6-be69a6cd1828\") " pod="openshift-multus/multus-2l8x5"
Oct 30 12:57:05.510221 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.510208    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-phkm5\" (UniqueName: \"kubernetes.io/projected/e1f629e6-6a96-444f-8329-e71bb98c9029-kube-api-access-phkm5\") pod \"router-default-649bdc978f-lkpqz\" (UID: \"e1f629e6-6a96-444f-8329-e71bb98c9029\") " pod="openshift-ingress/router-default-649bdc978f-lkpqz"
Oct 30 12:57:05.510801 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.510782    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gz9d6\" (UniqueName: \"kubernetes.io/projected/7ed65395-2cbd-46bf-81dc-fe307f3a9345-kube-api-access-gz9d6\") pod \"network-check-source-6f78d59f99-vdwgx\" (UID: \"7ed65395-2cbd-46bf-81dc-fe307f3a9345\") " pod="openshift-network-diagnostics/network-check-source-6f78d59f99-vdwgx"
Oct 30 12:57:05.510883 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.510868    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-66dlr\" (UniqueName: \"kubernetes.io/projected/56ff7ccd-573e-4bb2-a2b7-199a0346ed9f-kube-api-access-66dlr\") pod \"tuned-8mfkz\" (UID: \"56ff7ccd-573e-4bb2-a2b7-199a0346ed9f\") " pod="openshift-cluster-node-tuning-operator/tuned-8mfkz"
Oct 30 12:57:05.511334 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.511310    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9vr7r\" (UniqueName: \"kubernetes.io/projected/8d767e44-14df-4ea6-b391-bdf265773535-kube-api-access-9vr7r\") pod \"network-check-target-l5mdf\" (UID: \"8d767e44-14df-4ea6-b391-bdf265773535\") " pod="openshift-network-diagnostics/network-check-target-l5mdf"
Oct 30 12:57:05.511542 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.511528    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6rhvm\" (UniqueName: \"kubernetes.io/projected/a3ceb095-c75c-43b3-9de5-7dda644eddb1-kube-api-access-6rhvm\") pod \"aws-ebs-csi-driver-node-8bnjr\" (UID: \"a3ceb095-c75c-43b3-9de5-7dda644eddb1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8bnjr"
Oct 30 12:57:05.512241 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.512226    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zg85t\" (UniqueName: \"kubernetes.io/projected/55fa4842-3eb4-4e52-a4fc-18876ca409d5-kube-api-access-zg85t\") pod \"insights-runtime-extractor-knqz6\" (UID: \"55fa4842-3eb4-4e52-a4fc-18876ca409d5\") " pod="openshift-insights/insights-runtime-extractor-knqz6"
Oct 30 12:57:05.513556 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.513540    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vtsfh\" (UniqueName: \"kubernetes.io/projected/b11d3d97-ad88-4aa3-aaee-e83e3cf12f6d-kube-api-access-vtsfh\") pod \"network-metrics-daemon-p7tv6\" (UID: \"b11d3d97-ad88-4aa3-aaee-e83e3cf12f6d\") " pod="openshift-multus/network-metrics-daemon-p7tv6"
Oct 30 12:57:05.513974 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.513959    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hbsbr\" (UniqueName: \"kubernetes.io/projected/98e7e1f3-2ca6-45de-a5d8-d27c22e257b1-kube-api-access-hbsbr\") pod \"ingress-canary-bdf5s\" (UID: \"98e7e1f3-2ca6-45de-a5d8-d27c22e257b1\") " pod="openshift-ingress-canary/ingress-canary-bdf5s"
Oct 30 12:57:05.574949 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.574898    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4817ec63-f4d5-4420-9417-148346ff8bb5-host-run-ovn-kubernetes\") pod \"ovnkube-node-7rwcq\" (UID: \"4817ec63-f4d5-4420-9417-148346ff8bb5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7rwcq"
Oct 30 12:57:05.574949 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.574934    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/9a20c40d-f5f4-4d96-85b6-be69a6cd1828-host-run-multus-certs\") pod \"multus-2l8x5\" (UID: \"9a20c40d-f5f4-4d96-85b6-be69a6cd1828\") " pod="openshift-multus/multus-2l8x5"
Oct 30 12:57:05.575050 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.574963    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b11d3d97-ad88-4aa3-aaee-e83e3cf12f6d-metrics-certs\") pod \"network-metrics-daemon-p7tv6\" (UID: \"b11d3d97-ad88-4aa3-aaee-e83e3cf12f6d\") " pod="openshift-multus/network-metrics-daemon-p7tv6"
Oct 30 12:57:05.575050 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.574987    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4817ec63-f4d5-4420-9417-148346ff8bb5-var-lib-openvswitch\") pod \"ovnkube-node-7rwcq\" (UID: \"4817ec63-f4d5-4420-9417-148346ff8bb5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7rwcq"
Oct 30 12:57:05.575050 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.575000    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/9a20c40d-f5f4-4d96-85b6-be69a6cd1828-host-run-multus-certs\") pod \"multus-2l8x5\" (UID: \"9a20c40d-f5f4-4d96-85b6-be69a6cd1828\") " pod="openshift-multus/multus-2l8x5"
Oct 30 12:57:05.575050 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.575000    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4817ec63-f4d5-4420-9417-148346ff8bb5-host-run-ovn-kubernetes\") pod \"ovnkube-node-7rwcq\" (UID: \"4817ec63-f4d5-4420-9417-148346ff8bb5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7rwcq"
Oct 30 12:57:05.575050 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.575009    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/f660a256-a12d-4a4a-823d-4b0ff5dcb3da-dbus\") pod \"global-pull-secret-syncer-rpdcd\" (UID: \"f660a256-a12d-4a4a-823d-4b0ff5dcb3da\") " pod="kube-system/global-pull-secret-syncer-rpdcd"
Oct 30 12:57:05.575050 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.575045    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9962cf52-bd50-4a97-a699-4da03085eb81-cnibin\") pod \"multus-additional-cni-plugins-m4p7x\" (UID: \"9962cf52-bd50-4a97-a699-4da03085eb81\") " pod="openshift-multus/multus-additional-cni-plugins-m4p7x"
Oct 30 12:57:05.575244 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.575070    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/f660a256-a12d-4a4a-823d-4b0ff5dcb3da-dbus\") pod \"global-pull-secret-syncer-rpdcd\" (UID: \"f660a256-a12d-4a4a-823d-4b0ff5dcb3da\") " pod="kube-system/global-pull-secret-syncer-rpdcd"
Oct 30 12:57:05.575244 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.575078    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4817ec63-f4d5-4420-9417-148346ff8bb5-var-lib-openvswitch\") pod \"ovnkube-node-7rwcq\" (UID: \"4817ec63-f4d5-4420-9417-148346ff8bb5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7rwcq"
Oct 30 12:57:05.575244 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.575071    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9a20c40d-f5f4-4d96-85b6-be69a6cd1828-multus-conf-dir\") pod \"multus-2l8x5\" (UID: \"9a20c40d-f5f4-4d96-85b6-be69a6cd1828\") " pod="openshift-multus/multus-2l8x5"
Oct 30 12:57:05.575244 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.575104    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9962cf52-bd50-4a97-a699-4da03085eb81-cnibin\") pod \"multus-additional-cni-plugins-m4p7x\" (UID: \"9962cf52-bd50-4a97-a699-4da03085eb81\") " pod="openshift-multus/multus-additional-cni-plugins-m4p7x"
Oct 30 12:57:05.575244 ip-10-0-142-84 kubenswrapper[4943]: E1030 12:57:05.575096    4943 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Oct 30 12:57:05.575244 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.575129    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/55fa4842-3eb4-4e52-a4fc-18876ca409d5-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-knqz6\" (UID: \"55fa4842-3eb4-4e52-a4fc-18876ca409d5\") " pod="openshift-insights/insights-runtime-extractor-knqz6"
Oct 30 12:57:05.575244 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.575147    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/16402666-f516-49d1-be0d-f6ab7234e637-host\") pod \"node-ca-h8q28\" (UID: \"16402666-f516-49d1-be0d-f6ab7234e637\") " pod="openshift-image-registry/node-ca-h8q28"
Oct 30 12:57:05.575244 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.575163    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4817ec63-f4d5-4420-9417-148346ff8bb5-etc-openvswitch\") pod \"ovnkube-node-7rwcq\" (UID: \"4817ec63-f4d5-4420-9417-148346ff8bb5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7rwcq"
Oct 30 12:57:05.575244 ip-10-0-142-84 kubenswrapper[4943]: E1030 12:57:05.575173    4943 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b11d3d97-ad88-4aa3-aaee-e83e3cf12f6d-metrics-certs podName:b11d3d97-ad88-4aa3-aaee-e83e3cf12f6d nodeName:}" failed. No retries permitted until 2026-10-30 12:57:06.075159362 +0000 UTC m=+1.863163083 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b11d3d97-ad88-4aa3-aaee-e83e3cf12f6d-metrics-certs") pod "network-metrics-daemon-p7tv6" (UID: "b11d3d97-ad88-4aa3-aaee-e83e3cf12f6d") : secret "metrics-daemon-secret" not found
Oct 30 12:57:05.575244 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.575113    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9a20c40d-f5f4-4d96-85b6-be69a6cd1828-multus-conf-dir\") pod \"multus-2l8x5\" (UID: \"9a20c40d-f5f4-4d96-85b6-be69a6cd1828\") " pod="openshift-multus/multus-2l8x5"
Oct 30 12:57:05.575244 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.575193    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4817ec63-f4d5-4420-9417-148346ff8bb5-etc-openvswitch\") pod \"ovnkube-node-7rwcq\" (UID: \"4817ec63-f4d5-4420-9417-148346ff8bb5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7rwcq"
Oct 30 12:57:05.575244 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.575214    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/4883c26d-0877-4870-b0ee-406f5bfa8cf4-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-vsqgb\" (UID: \"4883c26d-0877-4870-b0ee-406f5bfa8cf4\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-vsqgb"
Oct 30 12:57:05.575244 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.575222    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/16402666-f516-49d1-be0d-f6ab7234e637-host\") pod \"node-ca-h8q28\" (UID: \"16402666-f516-49d1-be0d-f6ab7234e637\") " pod="openshift-image-registry/node-ca-h8q28"
Oct 30 12:57:05.575244 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.575236    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/56ff7ccd-573e-4bb2-a2b7-199a0346ed9f-lib-modules\") pod \"tuned-8mfkz\" (UID: \"56ff7ccd-573e-4bb2-a2b7-199a0346ed9f\") " pod="openshift-cluster-node-tuning-operator/tuned-8mfkz"
Oct 30 12:57:05.575244 ip-10-0-142-84 kubenswrapper[4943]: E1030 12:57:05.575243    4943 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Oct 30 12:57:05.575244 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.575252    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/56ff7ccd-573e-4bb2-a2b7-199a0346ed9f-var-lib-kubelet\") pod \"tuned-8mfkz\" (UID: \"56ff7ccd-573e-4bb2-a2b7-199a0346ed9f\") " pod="openshift-cluster-node-tuning-operator/tuned-8mfkz"
Oct 30 12:57:05.575828 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.575292    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/4817ec63-f4d5-4420-9417-148346ff8bb5-systemd-units\") pod \"ovnkube-node-7rwcq\" (UID: \"4817ec63-f4d5-4420-9417-148346ff8bb5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7rwcq"
Oct 30 12:57:05.575828 ip-10-0-142-84 kubenswrapper[4943]: E1030 12:57:05.575305    4943 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/55fa4842-3eb4-4e52-a4fc-18876ca409d5-insights-runtime-extractor-tls podName:55fa4842-3eb4-4e52-a4fc-18876ca409d5 nodeName:}" failed. No retries permitted until 2026-10-30 12:57:06.075290543 +0000 UTC m=+1.863294286 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/55fa4842-3eb4-4e52-a4fc-18876ca409d5-insights-runtime-extractor-tls") pod "insights-runtime-extractor-knqz6" (UID: "55fa4842-3eb4-4e52-a4fc-18876ca409d5") : secret "insights-runtime-extractor-tls" not found
Oct 30 12:57:05.575828 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.575322    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/4817ec63-f4d5-4420-9417-148346ff8bb5-systemd-units\") pod \"ovnkube-node-7rwcq\" (UID: \"4817ec63-f4d5-4420-9417-148346ff8bb5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7rwcq"
Oct 30 12:57:05.575828 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.575332    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/56ff7ccd-573e-4bb2-a2b7-199a0346ed9f-etc-sysctl-conf\") pod \"tuned-8mfkz\" (UID: \"56ff7ccd-573e-4bb2-a2b7-199a0346ed9f\") " pod="openshift-cluster-node-tuning-operator/tuned-8mfkz"
Oct 30 12:57:05.575828 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.575358    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/9a20c40d-f5f4-4d96-85b6-be69a6cd1828-multus-socket-dir-parent\") pod \"multus-2l8x5\" (UID: \"9a20c40d-f5f4-4d96-85b6-be69a6cd1828\") " pod="openshift-multus/multus-2l8x5"
Oct 30 12:57:05.575828 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.575365    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/56ff7ccd-573e-4bb2-a2b7-199a0346ed9f-var-lib-kubelet\") pod \"tuned-8mfkz\" (UID: \"56ff7ccd-573e-4bb2-a2b7-199a0346ed9f\") " pod="openshift-cluster-node-tuning-operator/tuned-8mfkz"
Oct 30 12:57:05.575828 ip-10-0-142-84 kubenswrapper[4943]: E1030 12:57:05.575380    4943 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Oct 30 12:57:05.575828 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.575388    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/56ff7ccd-573e-4bb2-a2b7-199a0346ed9f-lib-modules\") pod \"tuned-8mfkz\" (UID: \"56ff7ccd-573e-4bb2-a2b7-199a0346ed9f\") " pod="openshift-cluster-node-tuning-operator/tuned-8mfkz"
Oct 30 12:57:05.575828 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.575410    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/9a20c40d-f5f4-4d96-85b6-be69a6cd1828-multus-socket-dir-parent\") pod \"multus-2l8x5\" (UID: \"9a20c40d-f5f4-4d96-85b6-be69a6cd1828\") " pod="openshift-multus/multus-2l8x5"
Oct 30 12:57:05.575828 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.575415    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9a20c40d-f5f4-4d96-85b6-be69a6cd1828-host-var-lib-cni-bin\") pod \"multus-2l8x5\" (UID: \"9a20c40d-f5f4-4d96-85b6-be69a6cd1828\") " pod="openshift-multus/multus-2l8x5"
Oct 30 12:57:05.575828 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.575380    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9a20c40d-f5f4-4d96-85b6-be69a6cd1828-host-var-lib-cni-bin\") pod \"multus-2l8x5\" (UID: \"9a20c40d-f5f4-4d96-85b6-be69a6cd1828\") " pod="openshift-multus/multus-2l8x5"
Oct 30 12:57:05.575828 ip-10-0-142-84 kubenswrapper[4943]: E1030 12:57:05.575437    4943 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4883c26d-0877-4870-b0ee-406f5bfa8cf4-networking-console-plugin-cert podName:4883c26d-0877-4870-b0ee-406f5bfa8cf4 nodeName:}" failed. No retries permitted until 2026-10-30 12:57:06.075421286 +0000 UTC m=+1.863425013 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/4883c26d-0877-4870-b0ee-406f5bfa8cf4-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-vsqgb" (UID: "4883c26d-0877-4870-b0ee-406f5bfa8cf4") : secret "networking-console-plugin-cert" not found
Oct 30 12:57:05.575828 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.575474    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4817ec63-f4d5-4420-9417-148346ff8bb5-host-cni-bin\") pod \"ovnkube-node-7rwcq\" (UID: \"4817ec63-f4d5-4420-9417-148346ff8bb5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7rwcq"
Oct 30 12:57:05.575828 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.575482    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/56ff7ccd-573e-4bb2-a2b7-199a0346ed9f-etc-sysctl-conf\") pod \"tuned-8mfkz\" (UID: \"56ff7ccd-573e-4bb2-a2b7-199a0346ed9f\") " pod="openshift-cluster-node-tuning-operator/tuned-8mfkz"
Oct 30 12:57:05.575828 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.575497    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/56ff7ccd-573e-4bb2-a2b7-199a0346ed9f-etc-sysctl-d\") pod \"tuned-8mfkz\" (UID: \"56ff7ccd-573e-4bb2-a2b7-199a0346ed9f\") " pod="openshift-cluster-node-tuning-operator/tuned-8mfkz"
Oct 30 12:57:05.575828 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.575511    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4817ec63-f4d5-4420-9417-148346ff8bb5-host-cni-bin\") pod \"ovnkube-node-7rwcq\" (UID: \"4817ec63-f4d5-4420-9417-148346ff8bb5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7rwcq"
Oct 30 12:57:05.575828 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.575522    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/9a20c40d-f5f4-4d96-85b6-be69a6cd1828-host-run-k8s-cni-cncf-io\") pod \"multus-2l8x5\" (UID: \"9a20c40d-f5f4-4d96-85b6-be69a6cd1828\") " pod="openshift-multus/multus-2l8x5"
Oct 30 12:57:05.575828 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.575548    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/fcf99170-f5cf-4987-97bd-1c30b1b37a35-metrics-tls\") pod \"dns-default-g2ckh\" (UID: \"fcf99170-f5cf-4987-97bd-1c30b1b37a35\") " pod="openshift-dns/dns-default-g2ckh"
Oct 30 12:57:05.575828 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.575573    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/a3ceb095-c75c-43b3-9de5-7dda644eddb1-device-dir\") pod \"aws-ebs-csi-driver-node-8bnjr\" (UID: \"a3ceb095-c75c-43b3-9de5-7dda644eddb1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8bnjr"
Oct 30 12:57:05.575828 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.575576    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/9a20c40d-f5f4-4d96-85b6-be69a6cd1828-host-run-k8s-cni-cncf-io\") pod \"multus-2l8x5\" (UID: \"9a20c40d-f5f4-4d96-85b6-be69a6cd1828\") " pod="openshift-multus/multus-2l8x5"
Oct 30 12:57:05.575828 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.575609    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/56ff7ccd-573e-4bb2-a2b7-199a0346ed9f-etc-sysctl-d\") pod \"tuned-8mfkz\" (UID: \"56ff7ccd-573e-4bb2-a2b7-199a0346ed9f\") " pod="openshift-cluster-node-tuning-operator/tuned-8mfkz"
Oct 30 12:57:05.575828 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.575599    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/a3ceb095-c75c-43b3-9de5-7dda644eddb1-etc-selinux\") pod \"aws-ebs-csi-driver-node-8bnjr\" (UID: \"a3ceb095-c75c-43b3-9de5-7dda644eddb1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8bnjr"
Oct 30 12:57:05.575828 ip-10-0-142-84 kubenswrapper[4943]: E1030 12:57:05.575634    4943 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Oct 30 12:57:05.575828 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.575644    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/a3ceb095-c75c-43b3-9de5-7dda644eddb1-etc-selinux\") pod \"aws-ebs-csi-driver-node-8bnjr\" (UID: \"a3ceb095-c75c-43b3-9de5-7dda644eddb1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8bnjr"
Oct 30 12:57:05.575828 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.575649    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/56ff7ccd-573e-4bb2-a2b7-199a0346ed9f-etc-kubernetes\") pod \"tuned-8mfkz\" (UID: \"56ff7ccd-573e-4bb2-a2b7-199a0346ed9f\") " pod="openshift-cluster-node-tuning-operator/tuned-8mfkz"
Oct 30 12:57:05.575828 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.575633    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/a3ceb095-c75c-43b3-9de5-7dda644eddb1-device-dir\") pod \"aws-ebs-csi-driver-node-8bnjr\" (UID: \"a3ceb095-c75c-43b3-9de5-7dda644eddb1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8bnjr"
Oct 30 12:57:05.575828 ip-10-0-142-84 kubenswrapper[4943]: E1030 12:57:05.575675    4943 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fcf99170-f5cf-4987-97bd-1c30b1b37a35-metrics-tls podName:fcf99170-f5cf-4987-97bd-1c30b1b37a35 nodeName:}" failed. No retries permitted until 2026-10-30 12:57:06.07566464 +0000 UTC m=+1.863668374 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/fcf99170-f5cf-4987-97bd-1c30b1b37a35-metrics-tls") pod "dns-default-g2ckh" (UID: "fcf99170-f5cf-4987-97bd-1c30b1b37a35") : secret "dns-default-metrics-tls" not found
Oct 30 12:57:05.575828 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.575679    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/56ff7ccd-573e-4bb2-a2b7-199a0346ed9f-etc-kubernetes\") pod \"tuned-8mfkz\" (UID: \"56ff7ccd-573e-4bb2-a2b7-199a0346ed9f\") " pod="openshift-cluster-node-tuning-operator/tuned-8mfkz"
Oct 30 12:57:05.575828 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.575691    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/9a20c40d-f5f4-4d96-85b6-be69a6cd1828-host-var-lib-cni-multus\") pod \"multus-2l8x5\" (UID: \"9a20c40d-f5f4-4d96-85b6-be69a6cd1828\") " pod="openshift-multus/multus-2l8x5"
Oct 30 12:57:05.575828 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.575712    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/56ff7ccd-573e-4bb2-a2b7-199a0346ed9f-sys\") pod \"tuned-8mfkz\" (UID: \"56ff7ccd-573e-4bb2-a2b7-199a0346ed9f\") " pod="openshift-cluster-node-tuning-operator/tuned-8mfkz"
Oct 30 12:57:05.575828 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.575726    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b86e8e45-3c2d-46b6-9ade-d390b56e4a75-registry-tls\") pod \"image-registry-69c87564f5-n6wjk\" (UID: \"b86e8e45-3c2d-46b6-9ade-d390b56e4a75\") " pod="openshift-image-registry/image-registry-69c87564f5-n6wjk"
Oct 30 12:57:05.575828 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.575742    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/4817ec63-f4d5-4420-9417-148346ff8bb5-host-slash\") pod \"ovnkube-node-7rwcq\" (UID: \"4817ec63-f4d5-4420-9417-148346ff8bb5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7rwcq"
Oct 30 12:57:05.575828 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.575775    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/4817ec63-f4d5-4420-9417-148346ff8bb5-host-slash\") pod \"ovnkube-node-7rwcq\" (UID: \"4817ec63-f4d5-4420-9417-148346ff8bb5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7rwcq"
Oct 30 12:57:05.575828 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.575777    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/9a20c40d-f5f4-4d96-85b6-be69a6cd1828-host-var-lib-cni-multus\") pod \"multus-2l8x5\" (UID: \"9a20c40d-f5f4-4d96-85b6-be69a6cd1828\") " pod="openshift-multus/multus-2l8x5"
Oct 30 12:57:05.575828 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.575797    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4817ec63-f4d5-4420-9417-148346ff8bb5-host-run-netns\") pod \"ovnkube-node-7rwcq\" (UID: \"4817ec63-f4d5-4420-9417-148346ff8bb5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7rwcq"
Oct 30 12:57:05.575828 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.575802    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/56ff7ccd-573e-4bb2-a2b7-199a0346ed9f-sys\") pod \"tuned-8mfkz\" (UID: \"56ff7ccd-573e-4bb2-a2b7-199a0346ed9f\") " pod="openshift-cluster-node-tuning-operator/tuned-8mfkz"
Oct 30 12:57:05.575828 ip-10-0-142-84 kubenswrapper[4943]: E1030 12:57:05.575814    4943 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Oct 30 12:57:05.575828 ip-10-0-142-84 kubenswrapper[4943]: E1030 12:57:05.575824    4943 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-69c87564f5-n6wjk: secret "image-registry-tls" not found
Oct 30 12:57:05.575828 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.575831    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4817ec63-f4d5-4420-9417-148346ff8bb5-host-run-netns\") pod \"ovnkube-node-7rwcq\" (UID: \"4817ec63-f4d5-4420-9417-148346ff8bb5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7rwcq"
Oct 30 12:57:05.575828 ip-10-0-142-84 kubenswrapper[4943]: E1030 12:57:05.575853    4943 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Oct 30 12:57:05.577012 ip-10-0-142-84 kubenswrapper[4943]: E1030 12:57:05.575862    4943 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-77c6ff895d-vw6c6: secret "image-registry-tls" not found
Oct 30 12:57:05.577012 ip-10-0-142-84 kubenswrapper[4943]: E1030 12:57:05.575867    4943 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b86e8e45-3c2d-46b6-9ade-d390b56e4a75-registry-tls podName:b86e8e45-3c2d-46b6-9ade-d390b56e4a75 nodeName:}" failed. No retries permitted until 2026-10-30 12:57:06.075859393 +0000 UTC m=+1.863863133 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/b86e8e45-3c2d-46b6-9ade-d390b56e4a75-registry-tls") pod "image-registry-69c87564f5-n6wjk" (UID: "b86e8e45-3c2d-46b6-9ade-d390b56e4a75") : secret "image-registry-tls" not found
Oct 30 12:57:05.577012 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.575813    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1c405568-cb88-4485-ae2c-3614cc31ced6-registry-tls\") pod \"image-registry-77c6ff895d-vw6c6\" (UID: \"1c405568-cb88-4485-ae2c-3614cc31ced6\") " pod="openshift-image-registry/image-registry-77c6ff895d-vw6c6"
Oct 30 12:57:05.577012 ip-10-0-142-84 kubenswrapper[4943]: E1030 12:57:05.575884    4943 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1c405568-cb88-4485-ae2c-3614cc31ced6-registry-tls podName:1c405568-cb88-4485-ae2c-3614cc31ced6 nodeName:}" failed. No retries permitted until 2026-10-30 12:57:06.075876543 +0000 UTC m=+1.863880278 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/1c405568-cb88-4485-ae2c-3614cc31ced6-registry-tls") pod "image-registry-77c6ff895d-vw6c6" (UID: "1c405568-cb88-4485-ae2c-3614cc31ced6") : secret "image-registry-tls" not found
Oct 30 12:57:05.577012 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.575902    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/4817ec63-f4d5-4420-9417-148346ff8bb5-run-ovn\") pod \"ovnkube-node-7rwcq\" (UID: \"4817ec63-f4d5-4420-9417-148346ff8bb5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7rwcq"
Oct 30 12:57:05.577012 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.575918    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/19b38c46-bfd4-4ed0-a764-f7b672f18cc6-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-2sfj9\" (UID: \"19b38c46-bfd4-4ed0-a764-f7b672f18cc6\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-2sfj9"
Oct 30 12:57:05.577012 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.575934    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/a3ceb095-c75c-43b3-9de5-7dda644eddb1-registration-dir\") pod \"aws-ebs-csi-driver-node-8bnjr\" (UID: \"a3ceb095-c75c-43b3-9de5-7dda644eddb1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8bnjr"
Oct 30 12:57:05.577012 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.575948    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/a3ceb095-c75c-43b3-9de5-7dda644eddb1-sys-fs\") pod \"aws-ebs-csi-driver-node-8bnjr\" (UID: \"a3ceb095-c75c-43b3-9de5-7dda644eddb1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8bnjr"
Oct 30 12:57:05.577012 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.575961    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/56ff7ccd-573e-4bb2-a2b7-199a0346ed9f-etc-sysconfig\") pod \"tuned-8mfkz\" (UID: \"56ff7ccd-573e-4bb2-a2b7-199a0346ed9f\") " pod="openshift-cluster-node-tuning-operator/tuned-8mfkz"
Oct 30 12:57:05.577012 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.575975    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/9a20c40d-f5f4-4d96-85b6-be69a6cd1828-hostroot\") pod \"multus-2l8x5\" (UID: \"9a20c40d-f5f4-4d96-85b6-be69a6cd1828\") " pod="openshift-multus/multus-2l8x5"
Oct 30 12:57:05.577012 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.575989    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/4817ec63-f4d5-4420-9417-148346ff8bb5-node-log\") pod \"ovnkube-node-7rwcq\" (UID: \"4817ec63-f4d5-4420-9417-148346ff8bb5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7rwcq"
Oct 30 12:57:05.577012 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.576014    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/4817ec63-f4d5-4420-9417-148346ff8bb5-run-ovn\") pod \"ovnkube-node-7rwcq\" (UID: \"4817ec63-f4d5-4420-9417-148346ff8bb5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7rwcq"
Oct 30 12:57:05.577012 ip-10-0-142-84 kubenswrapper[4943]: E1030 12:57:05.576023    4943 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Oct 30 12:57:05.577012 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.576017    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/eca286e8-0b02-42af-9d15-1d8670b7362c-hosts-file\") pod \"node-resolver-4lfbk\" (UID: \"eca286e8-0b02-42af-9d15-1d8670b7362c\") " pod="openshift-dns/node-resolver-4lfbk"
Oct 30 12:57:05.577012 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.576036    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/a3ceb095-c75c-43b3-9de5-7dda644eddb1-sys-fs\") pod \"aws-ebs-csi-driver-node-8bnjr\" (UID: \"a3ceb095-c75c-43b3-9de5-7dda644eddb1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8bnjr"
Oct 30 12:57:05.577012 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.576046    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/56ff7ccd-573e-4bb2-a2b7-199a0346ed9f-etc-sysconfig\") pod \"tuned-8mfkz\" (UID: \"56ff7ccd-573e-4bb2-a2b7-199a0346ed9f\") " pod="openshift-cluster-node-tuning-operator/tuned-8mfkz"
Oct 30 12:57:05.577012 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.576051    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/9a20c40d-f5f4-4d96-85b6-be69a6cd1828-hostroot\") pod \"multus-2l8x5\" (UID: \"9a20c40d-f5f4-4d96-85b6-be69a6cd1828\") " pod="openshift-multus/multus-2l8x5"
Oct 30 12:57:05.577012 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.576055    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/a3ceb095-c75c-43b3-9de5-7dda644eddb1-socket-dir\") pod \"aws-ebs-csi-driver-node-8bnjr\" (UID: \"a3ceb095-c75c-43b3-9de5-7dda644eddb1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8bnjr"
Oct 30 12:57:05.577012 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.576068    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/a3ceb095-c75c-43b3-9de5-7dda644eddb1-registration-dir\") pod \"aws-ebs-csi-driver-node-8bnjr\" (UID: \"a3ceb095-c75c-43b3-9de5-7dda644eddb1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8bnjr"
Oct 30 12:57:05.577012 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.576071    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/eca286e8-0b02-42af-9d15-1d8670b7362c-hosts-file\") pod \"node-resolver-4lfbk\" (UID: \"eca286e8-0b02-42af-9d15-1d8670b7362c\") " pod="openshift-dns/node-resolver-4lfbk"
Oct 30 12:57:05.577012 ip-10-0-142-84 kubenswrapper[4943]: E1030 12:57:05.576080    4943 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/19b38c46-bfd4-4ed0-a764-f7b672f18cc6-cluster-monitoring-operator-tls podName:19b38c46-bfd4-4ed0-a764-f7b672f18cc6 nodeName:}" failed. No retries permitted until 2026-10-30 12:57:06.076068958 +0000 UTC m=+1.864072707 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/19b38c46-bfd4-4ed0-a764-f7b672f18cc6-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-2sfj9" (UID: "19b38c46-bfd4-4ed0-a764-f7b672f18cc6") : secret "cluster-monitoring-operator-tls" not found
Oct 30 12:57:05.577012 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.576095    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/4817ec63-f4d5-4420-9417-148346ff8bb5-node-log\") pod \"ovnkube-node-7rwcq\" (UID: \"4817ec63-f4d5-4420-9417-148346ff8bb5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7rwcq"
Oct 30 12:57:05.577012 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.576109    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/4817ec63-f4d5-4420-9417-148346ff8bb5-host-kubelet\") pod \"ovnkube-node-7rwcq\" (UID: \"4817ec63-f4d5-4420-9417-148346ff8bb5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7rwcq"
Oct 30 12:57:05.577012 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.576124    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/a3ceb095-c75c-43b3-9de5-7dda644eddb1-socket-dir\") pod \"aws-ebs-csi-driver-node-8bnjr\" (UID: \"a3ceb095-c75c-43b3-9de5-7dda644eddb1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8bnjr"
Oct 30 12:57:05.577012 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.576149    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4817ec63-f4d5-4420-9417-148346ff8bb5-run-openvswitch\") pod \"ovnkube-node-7rwcq\" (UID: \"4817ec63-f4d5-4420-9417-148346ff8bb5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7rwcq"
Oct 30 12:57:05.577012 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.576153    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/4817ec63-f4d5-4420-9417-148346ff8bb5-host-kubelet\") pod \"ovnkube-node-7rwcq\" (UID: \"4817ec63-f4d5-4420-9417-148346ff8bb5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7rwcq"
Oct 30 12:57:05.577012 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.576181    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9a20c40d-f5f4-4d96-85b6-be69a6cd1828-multus-cni-dir\") pod \"multus-2l8x5\" (UID: \"9a20c40d-f5f4-4d96-85b6-be69a6cd1828\") " pod="openshift-multus/multus-2l8x5"
Oct 30 12:57:05.577012 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.576208    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4817ec63-f4d5-4420-9417-148346ff8bb5-run-openvswitch\") pod \"ovnkube-node-7rwcq\" (UID: \"4817ec63-f4d5-4420-9417-148346ff8bb5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7rwcq"
Oct 30 12:57:05.577012 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.576216    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/55fa4842-3eb4-4e52-a4fc-18876ca409d5-crio-socket\") pod \"insights-runtime-extractor-knqz6\" (UID: \"55fa4842-3eb4-4e52-a4fc-18876ca409d5\") " pod="openshift-insights/insights-runtime-extractor-knqz6"
Oct 30 12:57:05.577012 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.576246    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9a20c40d-f5f4-4d96-85b6-be69a6cd1828-multus-cni-dir\") pod \"multus-2l8x5\" (UID: \"9a20c40d-f5f4-4d96-85b6-be69a6cd1828\") " pod="openshift-multus/multus-2l8x5"
Oct 30 12:57:05.577012 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.576307    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/4817ec63-f4d5-4420-9417-148346ff8bb5-run-systemd\") pod \"ovnkube-node-7rwcq\" (UID: \"4817ec63-f4d5-4420-9417-148346ff8bb5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7rwcq"
Oct 30 12:57:05.577012 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.576339    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/4817ec63-f4d5-4420-9417-148346ff8bb5-run-systemd\") pod \"ovnkube-node-7rwcq\" (UID: \"4817ec63-f4d5-4420-9417-148346ff8bb5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7rwcq"
Oct 30 12:57:05.577012 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.576343    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/55fa4842-3eb4-4e52-a4fc-18876ca409d5-crio-socket\") pod \"insights-runtime-extractor-knqz6\" (UID: \"55fa4842-3eb4-4e52-a4fc-18876ca409d5\") " pod="openshift-insights/insights-runtime-extractor-knqz6"
Oct 30 12:57:05.577012 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.576365    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e1f629e6-6a96-444f-8329-e71bb98c9029-service-ca-bundle\") pod \"router-default-649bdc978f-lkpqz\" (UID: \"e1f629e6-6a96-444f-8329-e71bb98c9029\") " pod="openshift-ingress/router-default-649bdc978f-lkpqz"
Oct 30 12:57:05.577012 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.576405    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e1f629e6-6a96-444f-8329-e71bb98c9029-metrics-certs\") pod \"router-default-649bdc978f-lkpqz\" (UID: \"e1f629e6-6a96-444f-8329-e71bb98c9029\") " pod="openshift-ingress/router-default-649bdc978f-lkpqz"
Oct 30 12:57:05.577012 ip-10-0-142-84 kubenswrapper[4943]: E1030 12:57:05.576468    4943 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e1f629e6-6a96-444f-8329-e71bb98c9029-service-ca-bundle podName:e1f629e6-6a96-444f-8329-e71bb98c9029 nodeName:}" failed. No retries permitted until 2026-10-30 12:57:06.07645929 +0000 UTC m=+1.864463021 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/e1f629e6-6a96-444f-8329-e71bb98c9029-service-ca-bundle") pod "router-default-649bdc978f-lkpqz" (UID: "e1f629e6-6a96-444f-8329-e71bb98c9029") : configmap references non-existent config key: service-ca.crt
Oct 30 12:57:05.577012 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.576496    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/4817ec63-f4d5-4420-9417-148346ff8bb5-host-cni-netd\") pod \"ovnkube-node-7rwcq\" (UID: \"4817ec63-f4d5-4420-9417-148346ff8bb5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7rwcq"
Oct 30 12:57:05.577012 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.576519    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/56ff7ccd-573e-4bb2-a2b7-199a0346ed9f-run\") pod \"tuned-8mfkz\" (UID: \"56ff7ccd-573e-4bb2-a2b7-199a0346ed9f\") " pod="openshift-cluster-node-tuning-operator/tuned-8mfkz"
Oct 30 12:57:05.577012 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.576533    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9a20c40d-f5f4-4d96-85b6-be69a6cd1828-host-var-lib-kubelet\") pod \"multus-2l8x5\" (UID: \"9a20c40d-f5f4-4d96-85b6-be69a6cd1828\") " pod="openshift-multus/multus-2l8x5"
Oct 30 12:57:05.577012 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.576550    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9a20c40d-f5f4-4d96-85b6-be69a6cd1828-host-run-netns\") pod \"multus-2l8x5\" (UID: \"9a20c40d-f5f4-4d96-85b6-be69a6cd1828\") " pod="openshift-multus/multus-2l8x5"
Oct 30 12:57:05.577012 ip-10-0-142-84 kubenswrapper[4943]: E1030 12:57:05.576500    4943 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Oct 30 12:57:05.577012 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.576574    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9962cf52-bd50-4a97-a699-4da03085eb81-system-cni-dir\") pod \"multus-additional-cni-plugins-m4p7x\" (UID: \"9962cf52-bd50-4a97-a699-4da03085eb81\") " pod="openshift-multus/multus-additional-cni-plugins-m4p7x"
Oct 30 12:57:05.577012 ip-10-0-142-84 kubenswrapper[4943]: E1030 12:57:05.576605    4943 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e1f629e6-6a96-444f-8329-e71bb98c9029-metrics-certs podName:e1f629e6-6a96-444f-8329-e71bb98c9029 nodeName:}" failed. No retries permitted until 2026-10-30 12:57:06.0765928 +0000 UTC m=+1.864596522 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e1f629e6-6a96-444f-8329-e71bb98c9029-metrics-certs") pod "router-default-649bdc978f-lkpqz" (UID: "e1f629e6-6a96-444f-8329-e71bb98c9029") : secret "router-metrics-certs-default" not found
Oct 30 12:57:05.577012 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.576601    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/56ff7ccd-573e-4bb2-a2b7-199a0346ed9f-run\") pod \"tuned-8mfkz\" (UID: \"56ff7ccd-573e-4bb2-a2b7-199a0346ed9f\") " pod="openshift-cluster-node-tuning-operator/tuned-8mfkz"
Oct 30 12:57:05.577012 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.576637    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9a20c40d-f5f4-4d96-85b6-be69a6cd1828-host-run-netns\") pod \"multus-2l8x5\" (UID: \"9a20c40d-f5f4-4d96-85b6-be69a6cd1828\") " pod="openshift-multus/multus-2l8x5"
Oct 30 12:57:05.577012 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.576603    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/4817ec63-f4d5-4420-9417-148346ff8bb5-host-cni-netd\") pod \"ovnkube-node-7rwcq\" (UID: \"4817ec63-f4d5-4420-9417-148346ff8bb5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7rwcq"
Oct 30 12:57:05.577012 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.576618    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9962cf52-bd50-4a97-a699-4da03085eb81-system-cni-dir\") pod \"multus-additional-cni-plugins-m4p7x\" (UID: \"9962cf52-bd50-4a97-a699-4da03085eb81\") " pod="openshift-multus/multus-additional-cni-plugins-m4p7x"
Oct 30 12:57:05.577012 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.576620    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/a3ceb095-c75c-43b3-9de5-7dda644eddb1-kubelet-dir\") pod \"aws-ebs-csi-driver-node-8bnjr\" (UID: \"a3ceb095-c75c-43b3-9de5-7dda644eddb1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8bnjr"
Oct 30 12:57:05.577012 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.576616    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9a20c40d-f5f4-4d96-85b6-be69a6cd1828-host-var-lib-kubelet\") pod \"multus-2l8x5\" (UID: \"9a20c40d-f5f4-4d96-85b6-be69a6cd1828\") " pod="openshift-multus/multus-2l8x5"
Oct 30 12:57:05.577012 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.576655    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/a3ceb095-c75c-43b3-9de5-7dda644eddb1-kubelet-dir\") pod \"aws-ebs-csi-driver-node-8bnjr\" (UID: \"a3ceb095-c75c-43b3-9de5-7dda644eddb1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8bnjr"
Oct 30 12:57:05.577012 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.576685    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4817ec63-f4d5-4420-9417-148346ff8bb5-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-7rwcq\" (UID: \"4817ec63-f4d5-4420-9417-148346ff8bb5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7rwcq"
Oct 30 12:57:05.577012 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.576707    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9a20c40d-f5f4-4d96-85b6-be69a6cd1828-cnibin\") pod \"multus-2l8x5\" (UID: \"9a20c40d-f5f4-4d96-85b6-be69a6cd1828\") " pod="openshift-multus/multus-2l8x5"
Oct 30 12:57:05.577012 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.576730    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9a20c40d-f5f4-4d96-85b6-be69a6cd1828-etc-kubernetes\") pod \"multus-2l8x5\" (UID: \"9a20c40d-f5f4-4d96-85b6-be69a6cd1828\") " pod="openshift-multus/multus-2l8x5"
Oct 30 12:57:05.577012 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.576753    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/55fa4842-3eb4-4e52-a4fc-18876ca409d5-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-knqz6\" (UID: \"55fa4842-3eb4-4e52-a4fc-18876ca409d5\") " pod="openshift-insights/insights-runtime-extractor-knqz6"
Oct 30 12:57:05.577012 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.576760    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4817ec63-f4d5-4420-9417-148346ff8bb5-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-7rwcq\" (UID: \"4817ec63-f4d5-4420-9417-148346ff8bb5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7rwcq"
Oct 30 12:57:05.577012 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.576781    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9a20c40d-f5f4-4d96-85b6-be69a6cd1828-cnibin\") pod \"multus-2l8x5\" (UID: \"9a20c40d-f5f4-4d96-85b6-be69a6cd1828\") " pod="openshift-multus/multus-2l8x5"
Oct 30 12:57:05.577012 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.576780    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/56ff7ccd-573e-4bb2-a2b7-199a0346ed9f-host\") pod \"tuned-8mfkz\" (UID: \"56ff7ccd-573e-4bb2-a2b7-199a0346ed9f\") " pod="openshift-cluster-node-tuning-operator/tuned-8mfkz"
Oct 30 12:57:05.577012 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.576785    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9a20c40d-f5f4-4d96-85b6-be69a6cd1828-etc-kubernetes\") pod \"multus-2l8x5\" (UID: \"9a20c40d-f5f4-4d96-85b6-be69a6cd1828\") " pod="openshift-multus/multus-2l8x5"
Oct 30 12:57:05.577012 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.576809    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ac1dc607-3072-4dbc-87e9-db316bf42dff-host-slash\") pod \"iptables-alerter-kb9lc\" (UID: \"ac1dc607-3072-4dbc-87e9-db316bf42dff\") " pod="openshift-network-operator/iptables-alerter-kb9lc"
Oct 30 12:57:05.577012 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.576824    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/56ff7ccd-573e-4bb2-a2b7-199a0346ed9f-host\") pod \"tuned-8mfkz\" (UID: \"56ff7ccd-573e-4bb2-a2b7-199a0346ed9f\") " pod="openshift-cluster-node-tuning-operator/tuned-8mfkz"
Oct 30 12:57:05.577012 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.576827    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/56ff7ccd-573e-4bb2-a2b7-199a0346ed9f-etc-modprobe-d\") pod \"tuned-8mfkz\" (UID: \"56ff7ccd-573e-4bb2-a2b7-199a0346ed9f\") " pod="openshift-cluster-node-tuning-operator/tuned-8mfkz"
Oct 30 12:57:05.577012 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.576851    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/4817ec63-f4d5-4420-9417-148346ff8bb5-log-socket\") pod \"ovnkube-node-7rwcq\" (UID: \"4817ec63-f4d5-4420-9417-148346ff8bb5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7rwcq"
Oct 30 12:57:05.577012 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.576858    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ac1dc607-3072-4dbc-87e9-db316bf42dff-host-slash\") pod \"iptables-alerter-kb9lc\" (UID: \"ac1dc607-3072-4dbc-87e9-db316bf42dff\") " pod="openshift-network-operator/iptables-alerter-kb9lc"
Oct 30 12:57:05.577012 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.576868    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/f660a256-a12d-4a4a-823d-4b0ff5dcb3da-kubelet-config\") pod \"global-pull-secret-syncer-rpdcd\" (UID: \"f660a256-a12d-4a4a-823d-4b0ff5dcb3da\") " pod="kube-system/global-pull-secret-syncer-rpdcd"
Oct 30 12:57:05.577012 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.576886    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/56ff7ccd-573e-4bb2-a2b7-199a0346ed9f-etc-modprobe-d\") pod \"tuned-8mfkz\" (UID: \"56ff7ccd-573e-4bb2-a2b7-199a0346ed9f\") " pod="openshift-cluster-node-tuning-operator/tuned-8mfkz"
Oct 30 12:57:05.577012 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.576893    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/f660a256-a12d-4a4a-823d-4b0ff5dcb3da-kubelet-config\") pod \"global-pull-secret-syncer-rpdcd\" (UID: \"f660a256-a12d-4a4a-823d-4b0ff5dcb3da\") " pod="kube-system/global-pull-secret-syncer-rpdcd"
Oct 30 12:57:05.577012 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.576894    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9a20c40d-f5f4-4d96-85b6-be69a6cd1828-system-cni-dir\") pod \"multus-2l8x5\" (UID: \"9a20c40d-f5f4-4d96-85b6-be69a6cd1828\") " pod="openshift-multus/multus-2l8x5"
Oct 30 12:57:05.577012 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.576914    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/4817ec63-f4d5-4420-9417-148346ff8bb5-log-socket\") pod \"ovnkube-node-7rwcq\" (UID: \"4817ec63-f4d5-4420-9417-148346ff8bb5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7rwcq"
Oct 30 12:57:05.577012 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.576919    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/56ff7ccd-573e-4bb2-a2b7-199a0346ed9f-etc-systemd\") pod \"tuned-8mfkz\" (UID: \"56ff7ccd-573e-4bb2-a2b7-199a0346ed9f\") " pod="openshift-cluster-node-tuning-operator/tuned-8mfkz"
Oct 30 12:57:05.577012 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.576945    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/23297e74-48f9-47b7-bb5a-80189bdbf2c4-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-7dtcd\" (UID: \"23297e74-48f9-47b7-bb5a-80189bdbf2c4\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-7dtcd"
Oct 30 12:57:05.577012 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.576951    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9a20c40d-f5f4-4d96-85b6-be69a6cd1828-system-cni-dir\") pod \"multus-2l8x5\" (UID: \"9a20c40d-f5f4-4d96-85b6-be69a6cd1828\") " pod="openshift-multus/multus-2l8x5"
Oct 30 12:57:05.577012 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.576977    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9962cf52-bd50-4a97-a699-4da03085eb81-tuning-conf-dir\") pod \"multus-additional-cni-plugins-m4p7x\" (UID: \"9962cf52-bd50-4a97-a699-4da03085eb81\") " pod="openshift-multus/multus-additional-cni-plugins-m4p7x"
Oct 30 12:57:05.577012 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.576992    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/56ff7ccd-573e-4bb2-a2b7-199a0346ed9f-etc-systemd\") pod \"tuned-8mfkz\" (UID: \"56ff7ccd-573e-4bb2-a2b7-199a0346ed9f\") " pod="openshift-cluster-node-tuning-operator/tuned-8mfkz"
Oct 30 12:57:05.577012 ip-10-0-142-84 kubenswrapper[4943]: E1030 12:57:05.577008    4943 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Oct 30 12:57:05.577012 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.577041    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9962cf52-bd50-4a97-a699-4da03085eb81-tuning-conf-dir\") pod \"multus-additional-cni-plugins-m4p7x\" (UID: \"9962cf52-bd50-4a97-a699-4da03085eb81\") " pod="openshift-multus/multus-additional-cni-plugins-m4p7x"
Oct 30 12:57:05.578986 ip-10-0-142-84 kubenswrapper[4943]: E1030 12:57:05.577059    4943 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/23297e74-48f9-47b7-bb5a-80189bdbf2c4-samples-operator-tls podName:23297e74-48f9-47b7-bb5a-80189bdbf2c4 nodeName:}" failed. No retries permitted until 2026-10-30 12:57:06.077044582 +0000 UTC m=+1.865048305 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/23297e74-48f9-47b7-bb5a-80189bdbf2c4-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-7dtcd" (UID: "23297e74-48f9-47b7-bb5a-80189bdbf2c4") : secret "samples-operator-tls" not found
Oct 30 12:57:05.578986 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.577212    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/55fa4842-3eb4-4e52-a4fc-18876ca409d5-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-knqz6\" (UID: \"55fa4842-3eb4-4e52-a4fc-18876ca409d5\") " pod="openshift-insights/insights-runtime-extractor-knqz6"
Oct 30 12:57:05.668920 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:05.668897    4943 scope.go:122] "RemoveContainer" containerID="0d08c313aba902fdd2b8020553685be737434486e076384118441c7db6d44b42"
Oct 30 12:57:06.081598 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:06.081570    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e1f629e6-6a96-444f-8329-e71bb98c9029-service-ca-bundle\") pod \"router-default-649bdc978f-lkpqz\" (UID: \"e1f629e6-6a96-444f-8329-e71bb98c9029\") " pod="openshift-ingress/router-default-649bdc978f-lkpqz"
Oct 30 12:57:06.081777 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:06.081604    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e1f629e6-6a96-444f-8329-e71bb98c9029-metrics-certs\") pod \"router-default-649bdc978f-lkpqz\" (UID: \"e1f629e6-6a96-444f-8329-e71bb98c9029\") " pod="openshift-ingress/router-default-649bdc978f-lkpqz"
Oct 30 12:57:06.081777 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:06.081658    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/23297e74-48f9-47b7-bb5a-80189bdbf2c4-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-7dtcd\" (UID: \"23297e74-48f9-47b7-bb5a-80189bdbf2c4\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-7dtcd"
Oct 30 12:57:06.081777 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:06.081693    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b11d3d97-ad88-4aa3-aaee-e83e3cf12f6d-metrics-certs\") pod \"network-metrics-daemon-p7tv6\" (UID: \"b11d3d97-ad88-4aa3-aaee-e83e3cf12f6d\") " pod="openshift-multus/network-metrics-daemon-p7tv6"
Oct 30 12:57:06.081777 ip-10-0-142-84 kubenswrapper[4943]: E1030 12:57:06.081733    4943 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e1f629e6-6a96-444f-8329-e71bb98c9029-service-ca-bundle podName:e1f629e6-6a96-444f-8329-e71bb98c9029 nodeName:}" failed. No retries permitted until 2026-10-30 12:57:07.081713881 +0000 UTC m=+2.869717598 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/e1f629e6-6a96-444f-8329-e71bb98c9029-service-ca-bundle") pod "router-default-649bdc978f-lkpqz" (UID: "e1f629e6-6a96-444f-8329-e71bb98c9029") : configmap references non-existent config key: service-ca.crt
Oct 30 12:57:06.081991 ip-10-0-142-84 kubenswrapper[4943]: E1030 12:57:06.081787    4943 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Oct 30 12:57:06.081991 ip-10-0-142-84 kubenswrapper[4943]: E1030 12:57:06.081793    4943 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Oct 30 12:57:06.081991 ip-10-0-142-84 kubenswrapper[4943]: E1030 12:57:06.081827    4943 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b11d3d97-ad88-4aa3-aaee-e83e3cf12f6d-metrics-certs podName:b11d3d97-ad88-4aa3-aaee-e83e3cf12f6d nodeName:}" failed. No retries permitted until 2026-10-30 12:57:07.081816729 +0000 UTC m=+2.869820437 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b11d3d97-ad88-4aa3-aaee-e83e3cf12f6d-metrics-certs") pod "network-metrics-daemon-p7tv6" (UID: "b11d3d97-ad88-4aa3-aaee-e83e3cf12f6d") : secret "metrics-daemon-secret" not found
Oct 30 12:57:06.081991 ip-10-0-142-84 kubenswrapper[4943]: E1030 12:57:06.081847    4943 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/23297e74-48f9-47b7-bb5a-80189bdbf2c4-samples-operator-tls podName:23297e74-48f9-47b7-bb5a-80189bdbf2c4 nodeName:}" failed. No retries permitted until 2026-10-30 12:57:07.08183552 +0000 UTC m=+2.869839249 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/23297e74-48f9-47b7-bb5a-80189bdbf2c4-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-7dtcd" (UID: "23297e74-48f9-47b7-bb5a-80189bdbf2c4") : secret "samples-operator-tls" not found
Oct 30 12:57:06.081991 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:06.081776    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/55fa4842-3eb4-4e52-a4fc-18876ca409d5-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-knqz6\" (UID: \"55fa4842-3eb4-4e52-a4fc-18876ca409d5\") " pod="openshift-insights/insights-runtime-extractor-knqz6"
Oct 30 12:57:06.081991 ip-10-0-142-84 kubenswrapper[4943]: E1030 12:57:06.081874    4943 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Oct 30 12:57:06.081991 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:06.081880    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/4883c26d-0877-4870-b0ee-406f5bfa8cf4-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-vsqgb\" (UID: \"4883c26d-0877-4870-b0ee-406f5bfa8cf4\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-vsqgb"
Oct 30 12:57:06.082337 ip-10-0-142-84 kubenswrapper[4943]: E1030 12:57:06.082314    4943 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Oct 30 12:57:06.082433 ip-10-0-142-84 kubenswrapper[4943]: E1030 12:57:06.082349    4943 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Oct 30 12:57:06.082433 ip-10-0-142-84 kubenswrapper[4943]: E1030 12:57:06.082322    4943 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e1f629e6-6a96-444f-8329-e71bb98c9029-metrics-certs podName:e1f629e6-6a96-444f-8329-e71bb98c9029 nodeName:}" failed. No retries permitted until 2026-10-30 12:57:07.081917692 +0000 UTC m=+2.869921400 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e1f629e6-6a96-444f-8329-e71bb98c9029-metrics-certs") pod "router-default-649bdc978f-lkpqz" (UID: "e1f629e6-6a96-444f-8329-e71bb98c9029") : secret "router-metrics-certs-default" not found
Oct 30 12:57:06.082530 ip-10-0-142-84 kubenswrapper[4943]: E1030 12:57:06.082435    4943 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/55fa4842-3eb4-4e52-a4fc-18876ca409d5-insights-runtime-extractor-tls podName:55fa4842-3eb4-4e52-a4fc-18876ca409d5 nodeName:}" failed. No retries permitted until 2026-10-30 12:57:07.082393312 +0000 UTC m=+2.870397034 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/55fa4842-3eb4-4e52-a4fc-18876ca409d5-insights-runtime-extractor-tls") pod "insights-runtime-extractor-knqz6" (UID: "55fa4842-3eb4-4e52-a4fc-18876ca409d5") : secret "insights-runtime-extractor-tls" not found
Oct 30 12:57:06.082583 ip-10-0-142-84 kubenswrapper[4943]: E1030 12:57:06.082560    4943 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4883c26d-0877-4870-b0ee-406f5bfa8cf4-networking-console-plugin-cert podName:4883c26d-0877-4870-b0ee-406f5bfa8cf4 nodeName:}" failed. No retries permitted until 2026-10-30 12:57:07.08252001 +0000 UTC m=+2.870523722 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/4883c26d-0877-4870-b0ee-406f5bfa8cf4-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-vsqgb" (UID: "4883c26d-0877-4870-b0ee-406f5bfa8cf4") : secret "networking-console-plugin-cert" not found
Oct 30 12:57:06.082643 ip-10-0-142-84 kubenswrapper[4943]: E1030 12:57:06.082629    4943 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Oct 30 12:57:06.082688 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:06.082491    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/fcf99170-f5cf-4987-97bd-1c30b1b37a35-metrics-tls\") pod \"dns-default-g2ckh\" (UID: \"fcf99170-f5cf-4987-97bd-1c30b1b37a35\") " pod="openshift-dns/dns-default-g2ckh"
Oct 30 12:57:06.082688 ip-10-0-142-84 kubenswrapper[4943]: E1030 12:57:06.082677    4943 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fcf99170-f5cf-4987-97bd-1c30b1b37a35-metrics-tls podName:fcf99170-f5cf-4987-97bd-1c30b1b37a35 nodeName:}" failed. No retries permitted until 2026-10-30 12:57:07.08266517 +0000 UTC m=+2.870668893 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/fcf99170-f5cf-4987-97bd-1c30b1b37a35-metrics-tls") pod "dns-default-g2ckh" (UID: "fcf99170-f5cf-4987-97bd-1c30b1b37a35") : secret "dns-default-metrics-tls" not found
Oct 30 12:57:06.082759 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:06.082724    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b86e8e45-3c2d-46b6-9ade-d390b56e4a75-registry-tls\") pod \"image-registry-69c87564f5-n6wjk\" (UID: \"b86e8e45-3c2d-46b6-9ade-d390b56e4a75\") " pod="openshift-image-registry/image-registry-69c87564f5-n6wjk"
Oct 30 12:57:06.082800 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:06.082779    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1c405568-cb88-4485-ae2c-3614cc31ced6-registry-tls\") pod \"image-registry-77c6ff895d-vw6c6\" (UID: \"1c405568-cb88-4485-ae2c-3614cc31ced6\") " pod="openshift-image-registry/image-registry-77c6ff895d-vw6c6"
Oct 30 12:57:06.082853 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:06.082820    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/19b38c46-bfd4-4ed0-a764-f7b672f18cc6-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-2sfj9\" (UID: \"19b38c46-bfd4-4ed0-a764-f7b672f18cc6\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-2sfj9"
Oct 30 12:57:06.082902 ip-10-0-142-84 kubenswrapper[4943]: E1030 12:57:06.082886    4943 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Oct 30 12:57:06.082954 ip-10-0-142-84 kubenswrapper[4943]: E1030 12:57:06.082901    4943 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-69c87564f5-n6wjk: secret "image-registry-tls" not found
Oct 30 12:57:06.083003 ip-10-0-142-84 kubenswrapper[4943]: E1030 12:57:06.082957    4943 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b86e8e45-3c2d-46b6-9ade-d390b56e4a75-registry-tls podName:b86e8e45-3c2d-46b6-9ade-d390b56e4a75 nodeName:}" failed. No retries permitted until 2026-10-30 12:57:07.082941957 +0000 UTC m=+2.870945680 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/b86e8e45-3c2d-46b6-9ade-d390b56e4a75-registry-tls") pod "image-registry-69c87564f5-n6wjk" (UID: "b86e8e45-3c2d-46b6-9ade-d390b56e4a75") : secret "image-registry-tls" not found
Oct 30 12:57:06.083064 ip-10-0-142-84 kubenswrapper[4943]: E1030 12:57:06.083043    4943 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Oct 30 12:57:06.083064 ip-10-0-142-84 kubenswrapper[4943]: E1030 12:57:06.083055    4943 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-77c6ff895d-vw6c6: secret "image-registry-tls" not found
Oct 30 12:57:06.083165 ip-10-0-142-84 kubenswrapper[4943]: E1030 12:57:06.083097    4943 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1c405568-cb88-4485-ae2c-3614cc31ced6-registry-tls podName:1c405568-cb88-4485-ae2c-3614cc31ced6 nodeName:}" failed. No retries permitted until 2026-10-30 12:57:07.083085158 +0000 UTC m=+2.871088881 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/1c405568-cb88-4485-ae2c-3614cc31ced6-registry-tls") pod "image-registry-77c6ff895d-vw6c6" (UID: "1c405568-cb88-4485-ae2c-3614cc31ced6") : secret "image-registry-tls" not found
Oct 30 12:57:06.083294 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:06.083254    4943 swap_util.go:78] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Oct 30 12:57:06.087084 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:06.087061    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/19b38c46-bfd4-4ed0-a764-f7b672f18cc6-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-2sfj9\" (UID: \"19b38c46-bfd4-4ed0-a764-f7b672f18cc6\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-2sfj9"
Oct 30 12:57:06.268436 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:06.268409    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-884576c57-2sfj9"
Oct 30 12:57:06.364013 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:06.363959    4943 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-zth9k"
Oct 30 12:57:06.397142 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:06.397120    4943 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-8fqfr_d8930ee7-0ae0-48b5-8ce3-cb8820355cef/console-operator/2.log"
Oct 30 12:57:06.397561 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:06.397543    4943 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-8fqfr_d8930ee7-0ae0-48b5-8ce3-cb8820355cef/console-operator/1.log"
Oct 30 12:57:06.397633 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:06.397580    4943 generic.go:356] "Generic (PLEG): container finished" podID="d8930ee7-0ae0-48b5-8ce3-cb8820355cef" containerID="6a29f45f9a0f97dd13834ffd12f3e6babdf2f98caa9c3dda57eb3443abb6aab7" exitCode=255
Oct 30 12:57:06.397682 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:06.397664    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7cd56b8dbc-8fqfr" event={"ID":"d8930ee7-0ae0-48b5-8ce3-cb8820355cef","Type":"ContainerDied","Data":"6a29f45f9a0f97dd13834ffd12f3e6babdf2f98caa9c3dda57eb3443abb6aab7"}
Oct 30 12:57:06.397730 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:06.397712    4943 scope.go:122] "RemoveContainer" containerID="0d08c313aba902fdd2b8020553685be737434486e076384118441c7db6d44b42"
Oct 30 12:57:06.398342 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:06.398326    4943 scope.go:122] "RemoveContainer" containerID="6a29f45f9a0f97dd13834ffd12f3e6babdf2f98caa9c3dda57eb3443abb6aab7"
Oct 30 12:57:06.398517 ip-10-0-142-84 kubenswrapper[4943]: E1030 12:57:06.398491    4943 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7cd56b8dbc-8fqfr_openshift-console-operator(d8930ee7-0ae0-48b5-8ce3-cb8820355cef)\"" pod="openshift-console-operator/console-operator-7cd56b8dbc-8fqfr" podUID="d8930ee7-0ae0-48b5-8ce3-cb8820355cef"
Oct 30 12:57:06.566791 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:06.566762    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-884576c57-2sfj9"]
Oct 30 12:57:06.569795 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:06.569771    4943 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod19b38c46_bfd4_4ed0_a764_f7b672f18cc6.slice/crio-bf73dfb0b3d9718fb911bb28f1fd80b0ecf6ae623442dedf6d8decc6cb40fa19 WatchSource:0}: Error finding container bf73dfb0b3d9718fb911bb28f1fd80b0ecf6ae623442dedf6d8decc6cb40fa19: Status 404 returned error can't find the container with id bf73dfb0b3d9718fb911bb28f1fd80b0ecf6ae623442dedf6d8decc6cb40fa19
Oct 30 12:57:06.571593 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:06.571574    4943 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 12:57:07.087838 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:07.087807    4943 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-7rwcq"
Oct 30 12:57:07.091810 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:07.091786    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/fcf99170-f5cf-4987-97bd-1c30b1b37a35-metrics-tls\") pod \"dns-default-g2ckh\" (UID: \"fcf99170-f5cf-4987-97bd-1c30b1b37a35\") " pod="openshift-dns/dns-default-g2ckh"
Oct 30 12:57:07.091927 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:07.091830    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b86e8e45-3c2d-46b6-9ade-d390b56e4a75-registry-tls\") pod \"image-registry-69c87564f5-n6wjk\" (UID: \"b86e8e45-3c2d-46b6-9ade-d390b56e4a75\") " pod="openshift-image-registry/image-registry-69c87564f5-n6wjk"
Oct 30 12:57:07.091927 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:07.091854    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1c405568-cb88-4485-ae2c-3614cc31ced6-registry-tls\") pod \"image-registry-77c6ff895d-vw6c6\" (UID: \"1c405568-cb88-4485-ae2c-3614cc31ced6\") " pod="openshift-image-registry/image-registry-77c6ff895d-vw6c6"
Oct 30 12:57:07.091927 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:07.091895    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e1f629e6-6a96-444f-8329-e71bb98c9029-service-ca-bundle\") pod \"router-default-649bdc978f-lkpqz\" (UID: \"e1f629e6-6a96-444f-8329-e71bb98c9029\") " pod="openshift-ingress/router-default-649bdc978f-lkpqz"
Oct 30 12:57:07.092092 ip-10-0-142-84 kubenswrapper[4943]: E1030 12:57:07.091929    4943 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Oct 30 12:57:07.092092 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:07.091951    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e1f629e6-6a96-444f-8329-e71bb98c9029-metrics-certs\") pod \"router-default-649bdc978f-lkpqz\" (UID: \"e1f629e6-6a96-444f-8329-e71bb98c9029\") " pod="openshift-ingress/router-default-649bdc978f-lkpqz"
Oct 30 12:57:07.092092 ip-10-0-142-84 kubenswrapper[4943]: E1030 12:57:07.091999    4943 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fcf99170-f5cf-4987-97bd-1c30b1b37a35-metrics-tls podName:fcf99170-f5cf-4987-97bd-1c30b1b37a35 nodeName:}" failed. No retries permitted until 2026-10-30 12:57:09.091980963 +0000 UTC m=+4.879984671 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/fcf99170-f5cf-4987-97bd-1c30b1b37a35-metrics-tls") pod "dns-default-g2ckh" (UID: "fcf99170-f5cf-4987-97bd-1c30b1b37a35") : secret "dns-default-metrics-tls" not found
Oct 30 12:57:07.092092 ip-10-0-142-84 kubenswrapper[4943]: E1030 12:57:07.092000    4943 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Oct 30 12:57:07.092092 ip-10-0-142-84 kubenswrapper[4943]: E1030 12:57:07.092018    4943 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-69c87564f5-n6wjk: secret "image-registry-tls" not found
Oct 30 12:57:07.092092 ip-10-0-142-84 kubenswrapper[4943]: E1030 12:57:07.092026    4943 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Oct 30 12:57:07.092092 ip-10-0-142-84 kubenswrapper[4943]: E1030 12:57:07.092037    4943 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-77c6ff895d-vw6c6: secret "image-registry-tls" not found
Oct 30 12:57:07.092092 ip-10-0-142-84 kubenswrapper[4943]: E1030 12:57:07.092047    4943 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Oct 30 12:57:07.092092 ip-10-0-142-84 kubenswrapper[4943]: E1030 12:57:07.092074    4943 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e1f629e6-6a96-444f-8329-e71bb98c9029-service-ca-bundle podName:e1f629e6-6a96-444f-8329-e71bb98c9029 nodeName:}" failed. No retries permitted until 2026-10-30 12:57:09.092052196 +0000 UTC m=+4.880055914 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/e1f629e6-6a96-444f-8329-e71bb98c9029-service-ca-bundle") pod "router-default-649bdc978f-lkpqz" (UID: "e1f629e6-6a96-444f-8329-e71bb98c9029") : configmap references non-existent config key: service-ca.crt
Oct 30 12:57:07.092092 ip-10-0-142-84 kubenswrapper[4943]: E1030 12:57:07.092096    4943 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b86e8e45-3c2d-46b6-9ade-d390b56e4a75-registry-tls podName:b86e8e45-3c2d-46b6-9ade-d390b56e4a75 nodeName:}" failed. No retries permitted until 2026-10-30 12:57:09.092088803 +0000 UTC m=+4.880092514 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/b86e8e45-3c2d-46b6-9ade-d390b56e4a75-registry-tls") pod "image-registry-69c87564f5-n6wjk" (UID: "b86e8e45-3c2d-46b6-9ade-d390b56e4a75") : secret "image-registry-tls" not found
Oct 30 12:57:07.092527 ip-10-0-142-84 kubenswrapper[4943]: E1030 12:57:07.092111    4943 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1c405568-cb88-4485-ae2c-3614cc31ced6-registry-tls podName:1c405568-cb88-4485-ae2c-3614cc31ced6 nodeName:}" failed. No retries permitted until 2026-10-30 12:57:09.092103469 +0000 UTC m=+4.880107176 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/1c405568-cb88-4485-ae2c-3614cc31ced6-registry-tls") pod "image-registry-77c6ff895d-vw6c6" (UID: "1c405568-cb88-4485-ae2c-3614cc31ced6") : secret "image-registry-tls" not found
Oct 30 12:57:07.092527 ip-10-0-142-84 kubenswrapper[4943]: E1030 12:57:07.092126    4943 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e1f629e6-6a96-444f-8329-e71bb98c9029-metrics-certs podName:e1f629e6-6a96-444f-8329-e71bb98c9029 nodeName:}" failed. No retries permitted until 2026-10-30 12:57:09.092119355 +0000 UTC m=+4.880123066 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e1f629e6-6a96-444f-8329-e71bb98c9029-metrics-certs") pod "router-default-649bdc978f-lkpqz" (UID: "e1f629e6-6a96-444f-8329-e71bb98c9029") : secret "router-metrics-certs-default" not found
Oct 30 12:57:07.092527 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:07.092193    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/23297e74-48f9-47b7-bb5a-80189bdbf2c4-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-7dtcd\" (UID: \"23297e74-48f9-47b7-bb5a-80189bdbf2c4\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-7dtcd"
Oct 30 12:57:07.092527 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:07.092238    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b11d3d97-ad88-4aa3-aaee-e83e3cf12f6d-metrics-certs\") pod \"network-metrics-daemon-p7tv6\" (UID: \"b11d3d97-ad88-4aa3-aaee-e83e3cf12f6d\") " pod="openshift-multus/network-metrics-daemon-p7tv6"
Oct 30 12:57:07.092527 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:07.092298    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/55fa4842-3eb4-4e52-a4fc-18876ca409d5-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-knqz6\" (UID: \"55fa4842-3eb4-4e52-a4fc-18876ca409d5\") " pod="openshift-insights/insights-runtime-extractor-knqz6"
Oct 30 12:57:07.092527 ip-10-0-142-84 kubenswrapper[4943]: E1030 12:57:07.092343    4943 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Oct 30 12:57:07.092527 ip-10-0-142-84 kubenswrapper[4943]: E1030 12:57:07.092354    4943 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Oct 30 12:57:07.092527 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:07.092339    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/4883c26d-0877-4870-b0ee-406f5bfa8cf4-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-vsqgb\" (UID: \"4883c26d-0877-4870-b0ee-406f5bfa8cf4\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-vsqgb"
Oct 30 12:57:07.092527 ip-10-0-142-84 kubenswrapper[4943]: E1030 12:57:07.092391    4943 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/23297e74-48f9-47b7-bb5a-80189bdbf2c4-samples-operator-tls podName:23297e74-48f9-47b7-bb5a-80189bdbf2c4 nodeName:}" failed. No retries permitted until 2026-10-30 12:57:09.092374071 +0000 UTC m=+4.880377794 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/23297e74-48f9-47b7-bb5a-80189bdbf2c4-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-7dtcd" (UID: "23297e74-48f9-47b7-bb5a-80189bdbf2c4") : secret "samples-operator-tls" not found
Oct 30 12:57:07.092527 ip-10-0-142-84 kubenswrapper[4943]: E1030 12:57:07.092413    4943 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b11d3d97-ad88-4aa3-aaee-e83e3cf12f6d-metrics-certs podName:b11d3d97-ad88-4aa3-aaee-e83e3cf12f6d nodeName:}" failed. No retries permitted until 2026-10-30 12:57:09.09240387 +0000 UTC m=+4.880407582 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b11d3d97-ad88-4aa3-aaee-e83e3cf12f6d-metrics-certs") pod "network-metrics-daemon-p7tv6" (UID: "b11d3d97-ad88-4aa3-aaee-e83e3cf12f6d") : secret "metrics-daemon-secret" not found
Oct 30 12:57:07.092527 ip-10-0-142-84 kubenswrapper[4943]: E1030 12:57:07.092438    4943 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Oct 30 12:57:07.092527 ip-10-0-142-84 kubenswrapper[4943]: E1030 12:57:07.092454    4943 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Oct 30 12:57:07.092527 ip-10-0-142-84 kubenswrapper[4943]: E1030 12:57:07.092481    4943 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4883c26d-0877-4870-b0ee-406f5bfa8cf4-networking-console-plugin-cert podName:4883c26d-0877-4870-b0ee-406f5bfa8cf4 nodeName:}" failed. No retries permitted until 2026-10-30 12:57:09.092472971 +0000 UTC m=+4.880476678 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/4883c26d-0877-4870-b0ee-406f5bfa8cf4-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-vsqgb" (UID: "4883c26d-0877-4870-b0ee-406f5bfa8cf4") : secret "networking-console-plugin-cert" not found
Oct 30 12:57:07.092527 ip-10-0-142-84 kubenswrapper[4943]: E1030 12:57:07.092503    4943 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/55fa4842-3eb4-4e52-a4fc-18876ca409d5-insights-runtime-extractor-tls podName:55fa4842-3eb4-4e52-a4fc-18876ca409d5 nodeName:}" failed. No retries permitted until 2026-10-30 12:57:09.092488185 +0000 UTC m=+4.880491896 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/55fa4842-3eb4-4e52-a4fc-18876ca409d5-insights-runtime-extractor-tls") pod "insights-runtime-extractor-knqz6" (UID: "55fa4842-3eb4-4e52-a4fc-18876ca409d5") : secret "insights-runtime-extractor-tls" not found
Oct 30 12:57:07.252609 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:07.252578    4943 kubelet.go:2727] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-7cd56b8dbc-8fqfr"
Oct 30 12:57:07.403459 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:07.403373    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-884576c57-2sfj9" event={"ID":"19b38c46-bfd4-4ed0-a764-f7b672f18cc6","Type":"ContainerStarted","Data":"bf73dfb0b3d9718fb911bb28f1fd80b0ecf6ae623442dedf6d8decc6cb40fa19"}
Oct 30 12:57:07.405024 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:07.405001    4943 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-8fqfr_d8930ee7-0ae0-48b5-8ce3-cb8820355cef/console-operator/2.log"
Oct 30 12:57:07.405374 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:07.405358    4943 scope.go:122] "RemoveContainer" containerID="6a29f45f9a0f97dd13834ffd12f3e6babdf2f98caa9c3dda57eb3443abb6aab7"
Oct 30 12:57:07.405585 ip-10-0-142-84 kubenswrapper[4943]: E1030 12:57:07.405567    4943 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7cd56b8dbc-8fqfr_openshift-console-operator(d8930ee7-0ae0-48b5-8ce3-cb8820355cef)\"" pod="openshift-console-operator/console-operator-7cd56b8dbc-8fqfr" podUID="d8930ee7-0ae0-48b5-8ce3-cb8820355cef"
Oct 30 12:57:07.729139 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:07.729106    4943 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-7rwcq"
Oct 30 12:57:07.729342 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:07.729327    4943 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Oct 30 12:57:08.409824 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:08.409750    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-884576c57-2sfj9" event={"ID":"19b38c46-bfd4-4ed0-a764-f7b672f18cc6","Type":"ContainerStarted","Data":"a844f9a675e9db7610631f613955774e89afd77939f69760cee2723b6d8a904e"}
Oct 30 12:57:09.112300 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:09.112244    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e1f629e6-6a96-444f-8329-e71bb98c9029-service-ca-bundle\") pod \"router-default-649bdc978f-lkpqz\" (UID: \"e1f629e6-6a96-444f-8329-e71bb98c9029\") " pod="openshift-ingress/router-default-649bdc978f-lkpqz"
Oct 30 12:57:09.112300 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:09.112304    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e1f629e6-6a96-444f-8329-e71bb98c9029-metrics-certs\") pod \"router-default-649bdc978f-lkpqz\" (UID: \"e1f629e6-6a96-444f-8329-e71bb98c9029\") " pod="openshift-ingress/router-default-649bdc978f-lkpqz"
Oct 30 12:57:09.112539 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:09.112342    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/23297e74-48f9-47b7-bb5a-80189bdbf2c4-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-7dtcd\" (UID: \"23297e74-48f9-47b7-bb5a-80189bdbf2c4\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-7dtcd"
Oct 30 12:57:09.112539 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:09.112366    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b11d3d97-ad88-4aa3-aaee-e83e3cf12f6d-metrics-certs\") pod \"network-metrics-daemon-p7tv6\" (UID: \"b11d3d97-ad88-4aa3-aaee-e83e3cf12f6d\") " pod="openshift-multus/network-metrics-daemon-p7tv6"
Oct 30 12:57:09.112539 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:09.112389    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/55fa4842-3eb4-4e52-a4fc-18876ca409d5-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-knqz6\" (UID: \"55fa4842-3eb4-4e52-a4fc-18876ca409d5\") " pod="openshift-insights/insights-runtime-extractor-knqz6"
Oct 30 12:57:09.112539 ip-10-0-142-84 kubenswrapper[4943]: E1030 12:57:09.112417    4943 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e1f629e6-6a96-444f-8329-e71bb98c9029-service-ca-bundle podName:e1f629e6-6a96-444f-8329-e71bb98c9029 nodeName:}" failed. No retries permitted until 2026-10-30 12:57:13.112395466 +0000 UTC m=+8.900399189 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/e1f629e6-6a96-444f-8329-e71bb98c9029-service-ca-bundle") pod "router-default-649bdc978f-lkpqz" (UID: "e1f629e6-6a96-444f-8329-e71bb98c9029") : configmap references non-existent config key: service-ca.crt
Oct 30 12:57:09.112539 ip-10-0-142-84 kubenswrapper[4943]: E1030 12:57:09.112455    4943 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Oct 30 12:57:09.112539 ip-10-0-142-84 kubenswrapper[4943]: E1030 12:57:09.112462    4943 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Oct 30 12:57:09.112539 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:09.112473    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/4883c26d-0877-4870-b0ee-406f5bfa8cf4-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-vsqgb\" (UID: \"4883c26d-0877-4870-b0ee-406f5bfa8cf4\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-vsqgb"
Oct 30 12:57:09.112539 ip-10-0-142-84 kubenswrapper[4943]: E1030 12:57:09.112490    4943 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Oct 30 12:57:09.112539 ip-10-0-142-84 kubenswrapper[4943]: E1030 12:57:09.112500    4943 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e1f629e6-6a96-444f-8329-e71bb98c9029-metrics-certs podName:e1f629e6-6a96-444f-8329-e71bb98c9029 nodeName:}" failed. No retries permitted until 2026-10-30 12:57:13.112486938 +0000 UTC m=+8.900490654 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e1f629e6-6a96-444f-8329-e71bb98c9029-metrics-certs") pod "router-default-649bdc978f-lkpqz" (UID: "e1f629e6-6a96-444f-8329-e71bb98c9029") : secret "router-metrics-certs-default" not found
Oct 30 12:57:09.112847 ip-10-0-142-84 kubenswrapper[4943]: E1030 12:57:09.112593    4943 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/55fa4842-3eb4-4e52-a4fc-18876ca409d5-insights-runtime-extractor-tls podName:55fa4842-3eb4-4e52-a4fc-18876ca409d5 nodeName:}" failed. No retries permitted until 2026-10-30 12:57:13.11255782 +0000 UTC m=+8.900561543 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/55fa4842-3eb4-4e52-a4fc-18876ca409d5-insights-runtime-extractor-tls") pod "insights-runtime-extractor-knqz6" (UID: "55fa4842-3eb4-4e52-a4fc-18876ca409d5") : secret "insights-runtime-extractor-tls" not found
Oct 30 12:57:09.112847 ip-10-0-142-84 kubenswrapper[4943]: E1030 12:57:09.112638    4943 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b11d3d97-ad88-4aa3-aaee-e83e3cf12f6d-metrics-certs podName:b11d3d97-ad88-4aa3-aaee-e83e3cf12f6d nodeName:}" failed. No retries permitted until 2026-10-30 12:57:13.112623477 +0000 UTC m=+8.900627201 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b11d3d97-ad88-4aa3-aaee-e83e3cf12f6d-metrics-certs") pod "network-metrics-daemon-p7tv6" (UID: "b11d3d97-ad88-4aa3-aaee-e83e3cf12f6d") : secret "metrics-daemon-secret" not found
Oct 30 12:57:09.112847 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:09.112695    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/fcf99170-f5cf-4987-97bd-1c30b1b37a35-metrics-tls\") pod \"dns-default-g2ckh\" (UID: \"fcf99170-f5cf-4987-97bd-1c30b1b37a35\") " pod="openshift-dns/dns-default-g2ckh"
Oct 30 12:57:09.112847 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:09.112758    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b86e8e45-3c2d-46b6-9ade-d390b56e4a75-registry-tls\") pod \"image-registry-69c87564f5-n6wjk\" (UID: \"b86e8e45-3c2d-46b6-9ade-d390b56e4a75\") " pod="openshift-image-registry/image-registry-69c87564f5-n6wjk"
Oct 30 12:57:09.112847 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:09.112798    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1c405568-cb88-4485-ae2c-3614cc31ced6-registry-tls\") pod \"image-registry-77c6ff895d-vw6c6\" (UID: \"1c405568-cb88-4485-ae2c-3614cc31ced6\") " pod="openshift-image-registry/image-registry-77c6ff895d-vw6c6"
Oct 30 12:57:09.112847 ip-10-0-142-84 kubenswrapper[4943]: E1030 12:57:09.112812    4943 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Oct 30 12:57:09.113043 ip-10-0-142-84 kubenswrapper[4943]: E1030 12:57:09.112871    4943 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fcf99170-f5cf-4987-97bd-1c30b1b37a35-metrics-tls podName:fcf99170-f5cf-4987-97bd-1c30b1b37a35 nodeName:}" failed. No retries permitted until 2026-10-30 12:57:13.112856676 +0000 UTC m=+8.900860383 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/fcf99170-f5cf-4987-97bd-1c30b1b37a35-metrics-tls") pod "dns-default-g2ckh" (UID: "fcf99170-f5cf-4987-97bd-1c30b1b37a35") : secret "dns-default-metrics-tls" not found
Oct 30 12:57:09.113043 ip-10-0-142-84 kubenswrapper[4943]: E1030 12:57:09.112900    4943 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Oct 30 12:57:09.113043 ip-10-0-142-84 kubenswrapper[4943]: E1030 12:57:09.112915    4943 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-69c87564f5-n6wjk: secret "image-registry-tls" not found
Oct 30 12:57:09.113043 ip-10-0-142-84 kubenswrapper[4943]: E1030 12:57:09.112953    4943 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b86e8e45-3c2d-46b6-9ade-d390b56e4a75-registry-tls podName:b86e8e45-3c2d-46b6-9ade-d390b56e4a75 nodeName:}" failed. No retries permitted until 2026-10-30 12:57:13.112940689 +0000 UTC m=+8.900944429 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/b86e8e45-3c2d-46b6-9ade-d390b56e4a75-registry-tls") pod "image-registry-69c87564f5-n6wjk" (UID: "b86e8e45-3c2d-46b6-9ade-d390b56e4a75") : secret "image-registry-tls" not found
Oct 30 12:57:09.113043 ip-10-0-142-84 kubenswrapper[4943]: E1030 12:57:09.112902    4943 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Oct 30 12:57:09.113043 ip-10-0-142-84 kubenswrapper[4943]: E1030 12:57:09.112972    4943 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-77c6ff895d-vw6c6: secret "image-registry-tls" not found
Oct 30 12:57:09.113043 ip-10-0-142-84 kubenswrapper[4943]: E1030 12:57:09.113003    4943 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1c405568-cb88-4485-ae2c-3614cc31ced6-registry-tls podName:1c405568-cb88-4485-ae2c-3614cc31ced6 nodeName:}" failed. No retries permitted until 2026-10-30 12:57:13.112992459 +0000 UTC m=+8.900996222 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/1c405568-cb88-4485-ae2c-3614cc31ced6-registry-tls") pod "image-registry-77c6ff895d-vw6c6" (UID: "1c405568-cb88-4485-ae2c-3614cc31ced6") : secret "image-registry-tls" not found
Oct 30 12:57:09.114786 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:09.114766    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/23297e74-48f9-47b7-bb5a-80189bdbf2c4-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-7dtcd\" (UID: \"23297e74-48f9-47b7-bb5a-80189bdbf2c4\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-7dtcd"
Oct 30 12:57:09.114829 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:09.114781    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/4883c26d-0877-4870-b0ee-406f5bfa8cf4-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-vsqgb\" (UID: \"4883c26d-0877-4870-b0ee-406f5bfa8cf4\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-vsqgb"
Oct 30 12:57:09.269404 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:09.269383    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-7dtcd"
Oct 30 12:57:09.272094 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:09.272076    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-86c478bb88-vsqgb"
Oct 30 12:57:09.304335 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:09.304310    4943 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7cd56b8dbc-8fqfr"
Oct 30 12:57:09.304736 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:09.304707    4943 scope.go:122] "RemoveContainer" containerID="6a29f45f9a0f97dd13834ffd12f3e6babdf2f98caa9c3dda57eb3443abb6aab7"
Oct 30 12:57:09.304948 ip-10-0-142-84 kubenswrapper[4943]: E1030 12:57:09.304927    4943 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7cd56b8dbc-8fqfr_openshift-console-operator(d8930ee7-0ae0-48b5-8ce3-cb8820355cef)\"" pod="openshift-console-operator/console-operator-7cd56b8dbc-8fqfr" podUID="d8930ee7-0ae0-48b5-8ce3-cb8820355cef"
Oct 30 12:57:09.414811 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:09.414775    4943 generic.go:356] "Generic (PLEG): container finished" podID="bd9e55af-64ad-4ffc-92c3-53b7bf654322" containerID="e77daa452696e3d2aec1881194c78a39904af7eed8db1d49200f93af317a9960" exitCode=0
Oct 30 12:57:09.415327 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:09.414850    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-7c4986d449-7cfvt" event={"ID":"bd9e55af-64ad-4ffc-92c3-53b7bf654322","Type":"ContainerDied","Data":"e77daa452696e3d2aec1881194c78a39904af7eed8db1d49200f93af317a9960"}
Oct 30 12:57:09.415414 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:09.415350    4943 scope.go:122] "RemoveContainer" containerID="e77daa452696e3d2aec1881194c78a39904af7eed8db1d49200f93af317a9960"
Oct 30 12:57:09.571941 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:09.571873    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-86c478bb88-vsqgb"]
Oct 30 12:57:09.574910 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:09.574887    4943 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4883c26d_0877_4870_b0ee_406f5bfa8cf4.slice/crio-61e752281b3321f48df580728fbea71cb24b3e76378b8551820ee96afe211a2e WatchSource:0}: Error finding container 61e752281b3321f48df580728fbea71cb24b3e76378b8551820ee96afe211a2e: Status 404 returned error can't find the container with id 61e752281b3321f48df580728fbea71cb24b3e76378b8551820ee96afe211a2e
Oct 30 12:57:09.586312 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:09.586288    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-7dtcd"]
Oct 30 12:57:09.605694 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:09.605670    4943 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-4lfbk_eca286e8-0b02-42af-9d15-1d8670b7362c/dns-node-resolver/0.log"
Oct 30 12:57:10.420523 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:10.420489    4943 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-l5mdf"
Oct 30 12:57:10.421005 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:10.420930    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-7c4986d449-7cfvt" event={"ID":"bd9e55af-64ad-4ffc-92c3-53b7bf654322","Type":"ContainerStarted","Data":"2866a14b88005f3ca49c5f98bfd7f492682aea58311f99d2d422c491262bf6dc"}
Oct 30 12:57:10.422567 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:10.422540    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-86c478bb88-vsqgb" event={"ID":"4883c26d-0877-4870-b0ee-406f5bfa8cf4","Type":"ContainerStarted","Data":"61e752281b3321f48df580728fbea71cb24b3e76378b8551820ee96afe211a2e"}
Oct 30 12:57:10.424115 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:10.424086    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-7dtcd" event={"ID":"23297e74-48f9-47b7-bb5a-80189bdbf2c4","Type":"ContainerStarted","Data":"1bff2e48e8858c69de01d11680e91a54be492a3cd1d5d23109933e4421452134"}
Oct 30 12:57:10.606543 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:10.606520    4943 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-h8q28_16402666-f516-49d1-be0d-f6ab7234e637/node-ca/0.log"
Oct 30 12:57:11.405912 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:11.405836    4943 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-bdf5s_98e7e1f3-2ca6-45de-a5d8-d27c22e257b1/serve-healthcheck-canary/0.log"
Oct 30 12:57:11.428997 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:11.428965    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-86c478bb88-vsqgb" event={"ID":"4883c26d-0877-4870-b0ee-406f5bfa8cf4","Type":"ContainerStarted","Data":"81b0192dbe2fb72ef0355a535b83ff7c767a1895ce35f6c078832783cb8e78bc"}
Oct 30 12:57:11.430671 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:11.430646    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-7dtcd" event={"ID":"23297e74-48f9-47b7-bb5a-80189bdbf2c4","Type":"ContainerStarted","Data":"6ee071a842a346eac6132f685bf1d0481d9a99c8f5771f5e60ecda81928af217"}
Oct 30 12:57:11.430770 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:11.430685    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-7dtcd" event={"ID":"23297e74-48f9-47b7-bb5a-80189bdbf2c4","Type":"ContainerStarted","Data":"7528b304f0c5774afb2ac37234a9f817e8ef7234e2a705943e680328ae8b462f"}
Oct 30 12:57:12.006663 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:12.006627    4943 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-66df5d496b-cfhqz_e30bf7f2-4c3a-4801-8ef1-f0611812a9d4/migrator/0.log"
Oct 30 12:57:12.207214 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:12.207181    4943 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-7d4bc45dd6-plmkt_85f4c399-0089-4399-86e2-069834f89286/kube-storage-version-migrator-operator/0.log"
Oct 30 12:57:13.150090 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:13.150053    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e1f629e6-6a96-444f-8329-e71bb98c9029-service-ca-bundle\") pod \"router-default-649bdc978f-lkpqz\" (UID: \"e1f629e6-6a96-444f-8329-e71bb98c9029\") " pod="openshift-ingress/router-default-649bdc978f-lkpqz"
Oct 30 12:57:13.150090 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:13.150087    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e1f629e6-6a96-444f-8329-e71bb98c9029-metrics-certs\") pod \"router-default-649bdc978f-lkpqz\" (UID: \"e1f629e6-6a96-444f-8329-e71bb98c9029\") " pod="openshift-ingress/router-default-649bdc978f-lkpqz"
Oct 30 12:57:13.150678 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:13.150123    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b11d3d97-ad88-4aa3-aaee-e83e3cf12f6d-metrics-certs\") pod \"network-metrics-daemon-p7tv6\" (UID: \"b11d3d97-ad88-4aa3-aaee-e83e3cf12f6d\") " pod="openshift-multus/network-metrics-daemon-p7tv6"
Oct 30 12:57:13.150678 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:13.150145    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/55fa4842-3eb4-4e52-a4fc-18876ca409d5-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-knqz6\" (UID: \"55fa4842-3eb4-4e52-a4fc-18876ca409d5\") " pod="openshift-insights/insights-runtime-extractor-knqz6"
Oct 30 12:57:13.150678 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:13.150177    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/fcf99170-f5cf-4987-97bd-1c30b1b37a35-metrics-tls\") pod \"dns-default-g2ckh\" (UID: \"fcf99170-f5cf-4987-97bd-1c30b1b37a35\") " pod="openshift-dns/dns-default-g2ckh"
Oct 30 12:57:13.150678 ip-10-0-142-84 kubenswrapper[4943]: E1030 12:57:13.150202    4943 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e1f629e6-6a96-444f-8329-e71bb98c9029-service-ca-bundle podName:e1f629e6-6a96-444f-8329-e71bb98c9029 nodeName:}" failed. No retries permitted until 2026-10-30 12:57:21.150186501 +0000 UTC m=+16.938190241 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/e1f629e6-6a96-444f-8329-e71bb98c9029-service-ca-bundle") pod "router-default-649bdc978f-lkpqz" (UID: "e1f629e6-6a96-444f-8329-e71bb98c9029") : configmap references non-existent config key: service-ca.crt
Oct 30 12:57:13.150678 ip-10-0-142-84 kubenswrapper[4943]: E1030 12:57:13.150313    4943 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Oct 30 12:57:13.150678 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:13.150333    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b86e8e45-3c2d-46b6-9ade-d390b56e4a75-registry-tls\") pod \"image-registry-69c87564f5-n6wjk\" (UID: \"b86e8e45-3c2d-46b6-9ade-d390b56e4a75\") " pod="openshift-image-registry/image-registry-69c87564f5-n6wjk"
Oct 30 12:57:13.150678 ip-10-0-142-84 kubenswrapper[4943]: E1030 12:57:13.150366    4943 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/55fa4842-3eb4-4e52-a4fc-18876ca409d5-insights-runtime-extractor-tls podName:55fa4842-3eb4-4e52-a4fc-18876ca409d5 nodeName:}" failed. No retries permitted until 2026-10-30 12:57:21.150351201 +0000 UTC m=+16.938354922 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/55fa4842-3eb4-4e52-a4fc-18876ca409d5-insights-runtime-extractor-tls") pod "insights-runtime-extractor-knqz6" (UID: "55fa4842-3eb4-4e52-a4fc-18876ca409d5") : secret "insights-runtime-extractor-tls" not found
Oct 30 12:57:13.150678 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:13.150389    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1c405568-cb88-4485-ae2c-3614cc31ced6-registry-tls\") pod \"image-registry-77c6ff895d-vw6c6\" (UID: \"1c405568-cb88-4485-ae2c-3614cc31ced6\") " pod="openshift-image-registry/image-registry-77c6ff895d-vw6c6"
Oct 30 12:57:13.152896 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:13.152868    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1c405568-cb88-4485-ae2c-3614cc31ced6-registry-tls\") pod \"image-registry-77c6ff895d-vw6c6\" (UID: \"1c405568-cb88-4485-ae2c-3614cc31ced6\") " pod="openshift-image-registry/image-registry-77c6ff895d-vw6c6"
Oct 30 12:57:13.153021 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:13.152901    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e1f629e6-6a96-444f-8329-e71bb98c9029-metrics-certs\") pod \"router-default-649bdc978f-lkpqz\" (UID: \"e1f629e6-6a96-444f-8329-e71bb98c9029\") " pod="openshift-ingress/router-default-649bdc978f-lkpqz"
Oct 30 12:57:13.153021 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:13.152921    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b86e8e45-3c2d-46b6-9ade-d390b56e4a75-registry-tls\") pod \"image-registry-69c87564f5-n6wjk\" (UID: \"b86e8e45-3c2d-46b6-9ade-d390b56e4a75\") " pod="openshift-image-registry/image-registry-69c87564f5-n6wjk"
Oct 30 12:57:13.153021 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:13.152945    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b11d3d97-ad88-4aa3-aaee-e83e3cf12f6d-metrics-certs\") pod \"network-metrics-daemon-p7tv6\" (UID: \"b11d3d97-ad88-4aa3-aaee-e83e3cf12f6d\") " pod="openshift-multus/network-metrics-daemon-p7tv6"
Oct 30 12:57:13.153021 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:13.152977    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/fcf99170-f5cf-4987-97bd-1c30b1b37a35-metrics-tls\") pod \"dns-default-g2ckh\" (UID: \"fcf99170-f5cf-4987-97bd-1c30b1b37a35\") " pod="openshift-dns/dns-default-g2ckh"
Oct 30 12:57:13.170004 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:13.169985    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-77c6ff895d-vw6c6"
Oct 30 12:57:13.174304 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:13.174289    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p7tv6"
Oct 30 12:57:13.180690 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:13.179888    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-69c87564f5-n6wjk"
Oct 30 12:57:13.180690 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:13.179976    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-g2ckh"
Oct 30 12:57:13.523089 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:13.522876    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-p7tv6"]
Oct 30 12:57:13.525684 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:13.525655    4943 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb11d3d97_ad88_4aa3_aaee_e83e3cf12f6d.slice/crio-8ad2d5eafb77468f2a21a3ebe951b52633d11c845c2bb1490039abe5428d96c0 WatchSource:0}: Error finding container 8ad2d5eafb77468f2a21a3ebe951b52633d11c845c2bb1490039abe5428d96c0: Status 404 returned error can't find the container with id 8ad2d5eafb77468f2a21a3ebe951b52633d11c845c2bb1490039abe5428d96c0
Oct 30 12:57:13.535423 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:13.535396    4943 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1c405568_cb88_4485_ae2c_3614cc31ced6.slice/crio-9d20cc83651a4a89afd1bcf30d72cadacbd043f7b11d1b93f276293ce0789b9f WatchSource:0}: Error finding container 9d20cc83651a4a89afd1bcf30d72cadacbd043f7b11d1b93f276293ce0789b9f: Status 404 returned error can't find the container with id 9d20cc83651a4a89afd1bcf30d72cadacbd043f7b11d1b93f276293ce0789b9f
Oct 30 12:57:13.536742 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:13.536712    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-77c6ff895d-vw6c6"]
Oct 30 12:57:13.545877 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:13.545850    4943 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb86e8e45_3c2d_46b6_9ade_d390b56e4a75.slice/crio-89790d76787d33e93317b53de75aba614499ee43ae7d9b7aac5e8f6c0f9723f6 WatchSource:0}: Error finding container 89790d76787d33e93317b53de75aba614499ee43ae7d9b7aac5e8f6c0f9723f6: Status 404 returned error can't find the container with id 89790d76787d33e93317b53de75aba614499ee43ae7d9b7aac5e8f6c0f9723f6
Oct 30 12:57:13.547383 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:13.547359    4943 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfcf99170_f5cf_4987_97bd_1c30b1b37a35.slice/crio-0fd5cc25b59919c89d219abd895d0c8e5c3b0610ec856afd31173d0785bff0fe WatchSource:0}: Error finding container 0fd5cc25b59919c89d219abd895d0c8e5c3b0610ec856afd31173d0785bff0fe: Status 404 returned error can't find the container with id 0fd5cc25b59919c89d219abd895d0c8e5c3b0610ec856afd31173d0785bff0fe
Oct 30 12:57:13.548138 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:13.548088    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-69c87564f5-n6wjk"]
Oct 30 12:57:13.553581 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:13.553555    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-g2ckh"]
Oct 30 12:57:14.442917 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:14.442886    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-77c6ff895d-vw6c6" event={"ID":"1c405568-cb88-4485-ae2c-3614cc31ced6","Type":"ContainerStarted","Data":"30a1965859c9d662e60abab43d07e297b337a36c46e94386a657899950cf843b"}
Oct 30 12:57:14.443312 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:14.442940    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-77c6ff895d-vw6c6" event={"ID":"1c405568-cb88-4485-ae2c-3614cc31ced6","Type":"ContainerStarted","Data":"9d20cc83651a4a89afd1bcf30d72cadacbd043f7b11d1b93f276293ce0789b9f"}
Oct 30 12:57:14.445044 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:14.445014    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-69c87564f5-n6wjk" event={"ID":"b86e8e45-3c2d-46b6-9ade-d390b56e4a75","Type":"ContainerStarted","Data":"960327b94c783e263c21e3c5d9ae67f87f458a09caab1213a12b9d9bbf238b0d"}
Oct 30 12:57:14.445149 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:14.445068    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-69c87564f5-n6wjk" event={"ID":"b86e8e45-3c2d-46b6-9ade-d390b56e4a75","Type":"ContainerStarted","Data":"89790d76787d33e93317b53de75aba614499ee43ae7d9b7aac5e8f6c0f9723f6"}
Oct 30 12:57:14.446442 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:14.446415    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-p7tv6" event={"ID":"b11d3d97-ad88-4aa3-aaee-e83e3cf12f6d","Type":"ContainerStarted","Data":"8ad2d5eafb77468f2a21a3ebe951b52633d11c845c2bb1490039abe5428d96c0"}
Oct 30 12:57:14.448612 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:14.448585    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-g2ckh" event={"ID":"fcf99170-f5cf-4987-97bd-1c30b1b37a35","Type":"ContainerStarted","Data":"0fd5cc25b59919c89d219abd895d0c8e5c3b0610ec856afd31173d0785bff0fe"}
Oct 30 12:57:14.974346 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:14.974312    4943 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-7rwcq"
Oct 30 12:57:15.454286 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:15.454235    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-p7tv6" event={"ID":"b11d3d97-ad88-4aa3-aaee-e83e3cf12f6d","Type":"ContainerStarted","Data":"c6bfb6b189a56fe654e8eeb1bb100b4ccbec2c1429c728b4a8918f80a1b7d5eb"}
Oct 30 12:57:15.454714 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:15.454404    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-p7tv6" event={"ID":"b11d3d97-ad88-4aa3-aaee-e83e3cf12f6d","Type":"ContainerStarted","Data":"1688185896a6b000fa9d3e5e4e9f759ccb3dd88dbe70cb6e3b1917ca75ff49ae"}
Oct 30 12:57:15.456141 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:15.456124    4943 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Oct 30 12:57:15.456252 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:15.456146    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-g2ckh" event={"ID":"fcf99170-f5cf-4987-97bd-1c30b1b37a35","Type":"ContainerStarted","Data":"2bad8ffc70276950fcadc8d01b635db85c1aff518b5dc407ea40d5b2678258b8"}
Oct 30 12:57:15.456252 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:15.456176    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-g2ckh" event={"ID":"fcf99170-f5cf-4987-97bd-1c30b1b37a35","Type":"ContainerStarted","Data":"5848df310cb1e9129fc2c59d89d69f36454665462e8c60b2575c597364f8bea5"}
Oct 30 12:57:15.456252 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:15.456188    4943 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Oct 30 12:57:15.456252 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:15.456218    4943 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-g2ckh"
Oct 30 12:57:15.879837 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:15.879806    4943 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-69c87564f5-n6wjk"
Oct 30 12:57:18.343279 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:18.343237    4943 scope.go:122] "RemoveContainer" containerID="6a29f45f9a0f97dd13834ffd12f3e6babdf2f98caa9c3dda57eb3443abb6aab7"
Oct 30 12:57:18.468139 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:18.468118    4943 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-8fqfr_d8930ee7-0ae0-48b5-8ce3-cb8820355cef/console-operator/2.log"
Oct 30 12:57:18.468302 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:18.468204    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7cd56b8dbc-8fqfr" event={"ID":"d8930ee7-0ae0-48b5-8ce3-cb8820355cef","Type":"ContainerStarted","Data":"12729c38f071afdcaebb5f3500d84cbb8193310db239dfc53777db6cbf1369ee"}
Oct 30 12:57:18.468519 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:18.468496    4943 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7cd56b8dbc-8fqfr"
Oct 30 12:57:18.469414 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:18.469388    4943 patch_prober.go:29] interesting pod/console-operator-7cd56b8dbc-8fqfr container/console-operator namespace/openshift-console-operator: Readiness probe status=failure output="Get \"https://10.132.0.14:8443/readyz\": dial tcp 10.132.0.14:8443: connect: connection refused" start-of-body=
Oct 30 12:57:18.469519 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:18.469444    4943 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-console-operator/console-operator-7cd56b8dbc-8fqfr" podUID="d8930ee7-0ae0-48b5-8ce3-cb8820355cef" containerName="console-operator" probeResult="failure" output="Get \"https://10.132.0.14:8443/readyz\": dial tcp 10.132.0.14:8443: connect: connection refused"
Oct 30 12:57:19.475703 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:19.475676    4943 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-7cd56b8dbc-8fqfr"
Oct 30 12:57:21.216549 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:21.216514    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e1f629e6-6a96-444f-8329-e71bb98c9029-service-ca-bundle\") pod \"router-default-649bdc978f-lkpqz\" (UID: \"e1f629e6-6a96-444f-8329-e71bb98c9029\") " pod="openshift-ingress/router-default-649bdc978f-lkpqz"
Oct 30 12:57:21.217038 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:21.216568    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/55fa4842-3eb4-4e52-a4fc-18876ca409d5-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-knqz6\" (UID: \"55fa4842-3eb4-4e52-a4fc-18876ca409d5\") " pod="openshift-insights/insights-runtime-extractor-knqz6"
Oct 30 12:57:21.217323 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:21.217297    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e1f629e6-6a96-444f-8329-e71bb98c9029-service-ca-bundle\") pod \"router-default-649bdc978f-lkpqz\" (UID: \"e1f629e6-6a96-444f-8329-e71bb98c9029\") " pod="openshift-ingress/router-default-649bdc978f-lkpqz"
Oct 30 12:57:21.218857 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:21.218838    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/55fa4842-3eb4-4e52-a4fc-18876ca409d5-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-knqz6\" (UID: \"55fa4842-3eb4-4e52-a4fc-18876ca409d5\") " pod="openshift-insights/insights-runtime-extractor-knqz6"
Oct 30 12:57:21.270485 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:21.270459    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-649bdc978f-lkpqz"
Oct 30 12:57:21.279695 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:21.279675    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-knqz6"
Oct 30 12:57:21.596639 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:21.596616    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-knqz6"]
Oct 30 12:57:21.599202 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:21.599170    4943 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod55fa4842_3eb4_4e52_a4fc_18876ca409d5.slice/crio-9dc099e320a8a56985b64b0989cb5a4d79a3e4dd6099edb6234e95bcde312ef8 WatchSource:0}: Error finding container 9dc099e320a8a56985b64b0989cb5a4d79a3e4dd6099edb6234e95bcde312ef8: Status 404 returned error can't find the container with id 9dc099e320a8a56985b64b0989cb5a4d79a3e4dd6099edb6234e95bcde312ef8
Oct 30 12:57:21.601868 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:21.601843    4943 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode1f629e6_6a96_444f_8329_e71bb98c9029.slice/crio-ecb63556b1b3ae6777f8fb57fc2e3df775b1a17567d5c7ab6b8fcbd8d9bc8327 WatchSource:0}: Error finding container ecb63556b1b3ae6777f8fb57fc2e3df775b1a17567d5c7ab6b8fcbd8d9bc8327: Status 404 returned error can't find the container with id ecb63556b1b3ae6777f8fb57fc2e3df775b1a17567d5c7ab6b8fcbd8d9bc8327
Oct 30 12:57:21.610477 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:21.610455    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-649bdc978f-lkpqz"]
Oct 30 12:57:22.488185 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:22.488152    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-649bdc978f-lkpqz" event={"ID":"e1f629e6-6a96-444f-8329-e71bb98c9029","Type":"ContainerStarted","Data":"e51b86d5bee27e7a19e7235bf04e4c083319c4b1748f0c06046252bd389132e4"}
Oct 30 12:57:22.488631 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:22.488239    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-649bdc978f-lkpqz" event={"ID":"e1f629e6-6a96-444f-8329-e71bb98c9029","Type":"ContainerStarted","Data":"ecb63556b1b3ae6777f8fb57fc2e3df775b1a17567d5c7ab6b8fcbd8d9bc8327"}
Oct 30 12:57:22.490003 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:22.489980    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-knqz6" event={"ID":"55fa4842-3eb4-4e52-a4fc-18876ca409d5","Type":"ContainerStarted","Data":"3481055a42f4fe906c2807f641ef185a90d8addbad5d72f54158395a8830342d"}
Oct 30 12:57:22.490104 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:22.490029    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-knqz6" event={"ID":"55fa4842-3eb4-4e52-a4fc-18876ca409d5","Type":"ContainerStarted","Data":"663860819b40271bc72cc90e56348533093c5e487c87e1a5433fd2bba4848309"}
Oct 30 12:57:22.490104 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:22.490056    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-knqz6" event={"ID":"55fa4842-3eb4-4e52-a4fc-18876ca409d5","Type":"ContainerStarted","Data":"9dc099e320a8a56985b64b0989cb5a4d79a3e4dd6099edb6234e95bcde312ef8"}
Oct 30 12:57:22.983529 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:22.983493    4943 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-77c6ff895d-vw6c6"
Oct 30 12:57:23.271358 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:23.271277    4943 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-649bdc978f-lkpqz"
Oct 30 12:57:23.274613 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:23.274341    4943 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-649bdc978f-lkpqz"
Oct 30 12:57:23.494847 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:23.494787    4943 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-649bdc978f-lkpqz"
Oct 30 12:57:23.496227 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:23.496204    4943 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-649bdc978f-lkpqz"
Oct 30 12:57:24.499928 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:24.499894    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-knqz6" event={"ID":"55fa4842-3eb4-4e52-a4fc-18876ca409d5","Type":"ContainerStarted","Data":"4919ec4d1d955a6549a12fc4e237509db808694f599b7eac048d90d83862c6c2"}
Oct 30 12:57:25.462764 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:25.462725    4943 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-g2ckh"
Oct 30 12:57:25.586291 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:25.586240    4943 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-77c6ff895d-vw6c6"]
Oct 30 12:57:27.122871 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:27.122841    4943 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-g2ckh_fcf99170-f5cf-4987-97bd-1c30b1b37a35/dns/0.log"
Oct 30 12:57:27.322418 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:27.322394    4943 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-g2ckh_fcf99170-f5cf-4987-97bd-1c30b1b37a35/kube-rbac-proxy/0.log"
Oct 30 12:57:27.921837 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:27.921811    4943 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-4lfbk_eca286e8-0b02-42af-9d15-1d8670b7362c/dns-node-resolver/0.log"
Oct 30 12:57:28.522094 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:28.522063    4943 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-69c87564f5-n6wjk_b86e8e45-3c2d-46b6-9ade-d390b56e4a75/registry/0.log"
Oct 30 12:57:28.922240 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:28.922168    4943 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-77c6ff895d-vw6c6_1c405568-cb88-4485-ae2c-3614cc31ced6/registry/0.log"
Oct 30 12:57:29.121336 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:29.121311    4943 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-h8q28_16402666-f516-49d1-be0d-f6ab7234e637/node-ca/0.log"
Oct 30 12:57:29.721982 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:29.721953    4943 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-649bdc978f-lkpqz_e1f629e6-6a96-444f-8329-e71bb98c9029/router/0.log"
Oct 30 12:57:29.921256 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:29.921225    4943 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-bdf5s_98e7e1f3-2ca6-45de-a5d8-d27c22e257b1/serve-healthcheck-canary/0.log"
Oct 30 12:57:30.423770 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:30.423739    4943 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-l5mdf"
Oct 30 12:57:33.872074 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:33.872042    4943 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-slf7p"]
Oct 30 12:57:33.888440 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:33.888416    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-slf7p"
Oct 30 12:57:33.908206 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:33.908179    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/ca443547-3eb9-4f7a-b85f-f4e1c87ea770-root\") pod \"node-exporter-slf7p\" (UID: \"ca443547-3eb9-4f7a-b85f-f4e1c87ea770\") " pod="openshift-monitoring/node-exporter-slf7p"
Oct 30 12:57:33.908361 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:33.908278    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/ca443547-3eb9-4f7a-b85f-f4e1c87ea770-node-exporter-tls\") pod \"node-exporter-slf7p\" (UID: \"ca443547-3eb9-4f7a-b85f-f4e1c87ea770\") " pod="openshift-monitoring/node-exporter-slf7p"
Oct 30 12:57:33.908361 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:33.908338    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ca443547-3eb9-4f7a-b85f-f4e1c87ea770-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-slf7p\" (UID: \"ca443547-3eb9-4f7a-b85f-f4e1c87ea770\") " pod="openshift-monitoring/node-exporter-slf7p"
Oct 30 12:57:33.908550 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:33.908384    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ca443547-3eb9-4f7a-b85f-f4e1c87ea770-sys\") pod \"node-exporter-slf7p\" (UID: \"ca443547-3eb9-4f7a-b85f-f4e1c87ea770\") " pod="openshift-monitoring/node-exporter-slf7p"
Oct 30 12:57:33.908550 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:33.908415    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/ca443547-3eb9-4f7a-b85f-f4e1c87ea770-node-exporter-accelerators-collector-config\") pod \"node-exporter-slf7p\" (UID: \"ca443547-3eb9-4f7a-b85f-f4e1c87ea770\") " pod="openshift-monitoring/node-exporter-slf7p"
Oct 30 12:57:33.908550 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:33.908433    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p9x9d\" (UniqueName: \"kubernetes.io/projected/ca443547-3eb9-4f7a-b85f-f4e1c87ea770-kube-api-access-p9x9d\") pod \"node-exporter-slf7p\" (UID: \"ca443547-3eb9-4f7a-b85f-f4e1c87ea770\") " pod="openshift-monitoring/node-exporter-slf7p"
Oct 30 12:57:33.908550 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:33.908514    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ca443547-3eb9-4f7a-b85f-f4e1c87ea770-metrics-client-ca\") pod \"node-exporter-slf7p\" (UID: \"ca443547-3eb9-4f7a-b85f-f4e1c87ea770\") " pod="openshift-monitoring/node-exporter-slf7p"
Oct 30 12:57:33.908725 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:33.908601    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/ca443547-3eb9-4f7a-b85f-f4e1c87ea770-node-exporter-textfile\") pod \"node-exporter-slf7p\" (UID: \"ca443547-3eb9-4f7a-b85f-f4e1c87ea770\") " pod="openshift-monitoring/node-exporter-slf7p"
Oct 30 12:57:33.908725 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:33.908701    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/ca443547-3eb9-4f7a-b85f-f4e1c87ea770-node-exporter-wtmp\") pod \"node-exporter-slf7p\" (UID: \"ca443547-3eb9-4f7a-b85f-f4e1c87ea770\") " pod="openshift-monitoring/node-exporter-slf7p"
Oct 30 12:57:33.909402 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:33.909383    4943 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Oct 30 12:57:33.922707 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:33.922673    4943 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Oct 30 12:57:33.978074 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:33.978043    4943 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Oct 30 12:57:33.978201 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:33.978142    4943 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Oct 30 12:57:33.978291 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:33.978276    4943 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-skpfr\""
Oct 30 12:57:34.009747 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:34.009718    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ca443547-3eb9-4f7a-b85f-f4e1c87ea770-sys\") pod \"node-exporter-slf7p\" (UID: \"ca443547-3eb9-4f7a-b85f-f4e1c87ea770\") " pod="openshift-monitoring/node-exporter-slf7p"
Oct 30 12:57:34.009884 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:34.009767    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/ca443547-3eb9-4f7a-b85f-f4e1c87ea770-node-exporter-accelerators-collector-config\") pod \"node-exporter-slf7p\" (UID: \"ca443547-3eb9-4f7a-b85f-f4e1c87ea770\") " pod="openshift-monitoring/node-exporter-slf7p"
Oct 30 12:57:34.009884 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:34.009823    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ca443547-3eb9-4f7a-b85f-f4e1c87ea770-sys\") pod \"node-exporter-slf7p\" (UID: \"ca443547-3eb9-4f7a-b85f-f4e1c87ea770\") " pod="openshift-monitoring/node-exporter-slf7p"
Oct 30 12:57:34.009884 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:34.009866    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p9x9d\" (UniqueName: \"kubernetes.io/projected/ca443547-3eb9-4f7a-b85f-f4e1c87ea770-kube-api-access-p9x9d\") pod \"node-exporter-slf7p\" (UID: \"ca443547-3eb9-4f7a-b85f-f4e1c87ea770\") " pod="openshift-monitoring/node-exporter-slf7p"
Oct 30 12:57:34.010053 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:34.009895    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ca443547-3eb9-4f7a-b85f-f4e1c87ea770-metrics-client-ca\") pod \"node-exporter-slf7p\" (UID: \"ca443547-3eb9-4f7a-b85f-f4e1c87ea770\") " pod="openshift-monitoring/node-exporter-slf7p"
Oct 30 12:57:34.010053 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:34.009912    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/ca443547-3eb9-4f7a-b85f-f4e1c87ea770-node-exporter-textfile\") pod \"node-exporter-slf7p\" (UID: \"ca443547-3eb9-4f7a-b85f-f4e1c87ea770\") " pod="openshift-monitoring/node-exporter-slf7p"
Oct 30 12:57:34.010053 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:34.009970    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/ca443547-3eb9-4f7a-b85f-f4e1c87ea770-node-exporter-wtmp\") pod \"node-exporter-slf7p\" (UID: \"ca443547-3eb9-4f7a-b85f-f4e1c87ea770\") " pod="openshift-monitoring/node-exporter-slf7p"
Oct 30 12:57:34.010053 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:34.010013    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/ca443547-3eb9-4f7a-b85f-f4e1c87ea770-root\") pod \"node-exporter-slf7p\" (UID: \"ca443547-3eb9-4f7a-b85f-f4e1c87ea770\") " pod="openshift-monitoring/node-exporter-slf7p"
Oct 30 12:57:34.010053 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:34.010036    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/ca443547-3eb9-4f7a-b85f-f4e1c87ea770-node-exporter-tls\") pod \"node-exporter-slf7p\" (UID: \"ca443547-3eb9-4f7a-b85f-f4e1c87ea770\") " pod="openshift-monitoring/node-exporter-slf7p"
Oct 30 12:57:34.010334 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:34.010071    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ca443547-3eb9-4f7a-b85f-f4e1c87ea770-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-slf7p\" (UID: \"ca443547-3eb9-4f7a-b85f-f4e1c87ea770\") " pod="openshift-monitoring/node-exporter-slf7p"
Oct 30 12:57:34.010334 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:34.010087    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/ca443547-3eb9-4f7a-b85f-f4e1c87ea770-root\") pod \"node-exporter-slf7p\" (UID: \"ca443547-3eb9-4f7a-b85f-f4e1c87ea770\") " pod="openshift-monitoring/node-exporter-slf7p"
Oct 30 12:57:34.010334 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:34.010140    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/ca443547-3eb9-4f7a-b85f-f4e1c87ea770-node-exporter-wtmp\") pod \"node-exporter-slf7p\" (UID: \"ca443547-3eb9-4f7a-b85f-f4e1c87ea770\") " pod="openshift-monitoring/node-exporter-slf7p"
Oct 30 12:57:34.010334 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:34.010192    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/ca443547-3eb9-4f7a-b85f-f4e1c87ea770-node-exporter-textfile\") pod \"node-exporter-slf7p\" (UID: \"ca443547-3eb9-4f7a-b85f-f4e1c87ea770\") " pod="openshift-monitoring/node-exporter-slf7p"
Oct 30 12:57:34.010532 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:34.010445    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/ca443547-3eb9-4f7a-b85f-f4e1c87ea770-node-exporter-accelerators-collector-config\") pod \"node-exporter-slf7p\" (UID: \"ca443547-3eb9-4f7a-b85f-f4e1c87ea770\") " pod="openshift-monitoring/node-exporter-slf7p"
Oct 30 12:57:34.010532 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:34.010512    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ca443547-3eb9-4f7a-b85f-f4e1c87ea770-metrics-client-ca\") pod \"node-exporter-slf7p\" (UID: \"ca443547-3eb9-4f7a-b85f-f4e1c87ea770\") " pod="openshift-monitoring/node-exporter-slf7p"
Oct 30 12:57:34.012409 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:34.012391    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/ca443547-3eb9-4f7a-b85f-f4e1c87ea770-node-exporter-tls\") pod \"node-exporter-slf7p\" (UID: \"ca443547-3eb9-4f7a-b85f-f4e1c87ea770\") " pod="openshift-monitoring/node-exporter-slf7p"
Oct 30 12:57:34.012471 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:34.012415    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ca443547-3eb9-4f7a-b85f-f4e1c87ea770-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-slf7p\" (UID: \"ca443547-3eb9-4f7a-b85f-f4e1c87ea770\") " pod="openshift-monitoring/node-exporter-slf7p"
Oct 30 12:57:34.035498 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:34.035472    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p9x9d\" (UniqueName: \"kubernetes.io/projected/ca443547-3eb9-4f7a-b85f-f4e1c87ea770-kube-api-access-p9x9d\") pod \"node-exporter-slf7p\" (UID: \"ca443547-3eb9-4f7a-b85f-f4e1c87ea770\") " pod="openshift-monitoring/node-exporter-slf7p"
Oct 30 12:57:34.234731 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:34.234703    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-slf7p"
Oct 30 12:57:34.268923 ip-10-0-142-84 kubenswrapper[4943]: W1030 12:57:34.268888    4943 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podca443547_3eb9_4f7a_b85f_f4e1c87ea770.slice/crio-935b8af803733867bef5a02795e2ccdbcd74334b3622ff4459266572ce327d5a WatchSource:0}: Error finding container 935b8af803733867bef5a02795e2ccdbcd74334b3622ff4459266572ce327d5a: Status 404 returned error can't find the container with id 935b8af803733867bef5a02795e2ccdbcd74334b3622ff4459266572ce327d5a
Oct 30 12:57:34.539947 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:34.539861    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-slf7p" event={"ID":"ca443547-3eb9-4f7a-b85f-f4e1c87ea770","Type":"ContainerStarted","Data":"935b8af803733867bef5a02795e2ccdbcd74334b3622ff4459266572ce327d5a"}
Oct 30 12:57:35.592229 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:35.592208    4943 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-77c6ff895d-vw6c6"
Oct 30 12:57:35.955621 ip-10-0-142-84 kubenswrapper[4943]: E1030 12:57:35.955587    4943 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-knqz6" containerName="extractor"
Oct 30 12:57:36.464947 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:36.464921    4943 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-69c87564f5-n6wjk"
Oct 30 12:57:36.549684 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:36.549569    4943 generic.go:356] "Generic (PLEG): container finished" podID="ca443547-3eb9-4f7a-b85f-f4e1c87ea770" containerID="bb68ca320ac07e9d6cca9199798c3ef469bf57dc42f2853a2764e9a1c12ec71a" exitCode=0
Oct 30 12:57:36.549684 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:36.549627    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-slf7p" event={"ID":"ca443547-3eb9-4f7a-b85f-f4e1c87ea770","Type":"ContainerDied","Data":"bb68ca320ac07e9d6cca9199798c3ef469bf57dc42f2853a2764e9a1c12ec71a"}
Oct 30 12:57:37.556187 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:37.556150    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-slf7p" event={"ID":"ca443547-3eb9-4f7a-b85f-f4e1c87ea770","Type":"ContainerStarted","Data":"ba9d2a72d7144aa09208fe53642a7288e07f7896a856c909228fd8b8d3e5f5ae"}
Oct 30 12:57:37.556665 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:37.556202    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-slf7p" event={"ID":"ca443547-3eb9-4f7a-b85f-f4e1c87ea770","Type":"ContainerStarted","Data":"9e35e861d6b2391927b95f8a698859af4e8139794834583e431418cb6d5b96d7"}
Oct 30 12:57:37.612689 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:37.612627    4943 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-slf7p" podStartSLOduration=3.299671828 podStartE2EDuration="4.612611591s" totalImagesPullingTime="1.231105778s" totalInitContainerRuntime="81.833985ms" isStatefulPod=true podCreationTimestamp="2026-10-30 12:57:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 12:57:37.611498482 +0000 UTC m=+33.399502212" watchObservedRunningTime="2026-10-30 12:57:37.612611591 +0000 UTC m=+33.400615321"
Oct 30 12:57:49.133485 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:49.133403    4943 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-69c87564f5-n6wjk"]
Oct 30 12:57:50.606389 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:50.606328    4943 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-image-registry/image-registry-77c6ff895d-vw6c6" podUID="1c405568-cb88-4485-ae2c-3614cc31ced6" containerName="registry" containerID="cri-o://30a1965859c9d662e60abab43d07e297b337a36c46e94386a657899950cf843b" gracePeriod=30
Oct 30 12:57:50.843213 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:50.843184    4943 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-77c6ff895d-vw6c6"
Oct 30 12:57:50.953212 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:50.953186    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1c405568-cb88-4485-ae2c-3614cc31ced6-kube-api-access-54kft\" (UniqueName: \"kubernetes.io/projected/1c405568-cb88-4485-ae2c-3614cc31ced6-kube-api-access-54kft\") pod \"1c405568-cb88-4485-ae2c-3614cc31ced6\" (UID: \"1c405568-cb88-4485-ae2c-3614cc31ced6\") "
Oct 30 12:57:50.953392 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:50.953230    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/1c405568-cb88-4485-ae2c-3614cc31ced6-image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/1c405568-cb88-4485-ae2c-3614cc31ced6-image-registry-private-configuration\") pod \"1c405568-cb88-4485-ae2c-3614cc31ced6\" (UID: \"1c405568-cb88-4485-ae2c-3614cc31ced6\") "
Oct 30 12:57:50.953392 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:50.953250    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/1c405568-cb88-4485-ae2c-3614cc31ced6-installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/1c405568-cb88-4485-ae2c-3614cc31ced6-installation-pull-secrets\") pod \"1c405568-cb88-4485-ae2c-3614cc31ced6\" (UID: \"1c405568-cb88-4485-ae2c-3614cc31ced6\") "
Oct 30 12:57:50.953392 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:50.953324    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1c405568-cb88-4485-ae2c-3614cc31ced6-bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1c405568-cb88-4485-ae2c-3614cc31ced6-bound-sa-token\") pod \"1c405568-cb88-4485-ae2c-3614cc31ced6\" (UID: \"1c405568-cb88-4485-ae2c-3614cc31ced6\") "
Oct 30 12:57:50.953392 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:50.953368    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1c405568-cb88-4485-ae2c-3614cc31ced6-registry-certificates\" (UniqueName: \"kubernetes.io/configmap/1c405568-cb88-4485-ae2c-3614cc31ced6-registry-certificates\") pod \"1c405568-cb88-4485-ae2c-3614cc31ced6\" (UID: \"1c405568-cb88-4485-ae2c-3614cc31ced6\") "
Oct 30 12:57:50.953392 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:50.953386    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1c405568-cb88-4485-ae2c-3614cc31ced6-registry-tls\" (UniqueName: \"kubernetes.io/projected/1c405568-cb88-4485-ae2c-3614cc31ced6-registry-tls\") pod \"1c405568-cb88-4485-ae2c-3614cc31ced6\" (UID: \"1c405568-cb88-4485-ae2c-3614cc31ced6\") "
Oct 30 12:57:50.953661 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:50.953433    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1c405568-cb88-4485-ae2c-3614cc31ced6-trusted-ca\" (UniqueName: \"kubernetes.io/configmap/1c405568-cb88-4485-ae2c-3614cc31ced6-trusted-ca\") pod \"1c405568-cb88-4485-ae2c-3614cc31ced6\" (UID: \"1c405568-cb88-4485-ae2c-3614cc31ced6\") "
Oct 30 12:57:50.953661 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:50.953475    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1c405568-cb88-4485-ae2c-3614cc31ced6-ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/1c405568-cb88-4485-ae2c-3614cc31ced6-ca-trust-extracted\") pod \"1c405568-cb88-4485-ae2c-3614cc31ced6\" (UID: \"1c405568-cb88-4485-ae2c-3614cc31ced6\") "
Oct 30 12:57:50.953996 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:50.953929    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1c405568-cb88-4485-ae2c-3614cc31ced6-registry-certificates" pod "1c405568-cb88-4485-ae2c-3614cc31ced6" (UID: "1c405568-cb88-4485-ae2c-3614cc31ced6"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 12:57:50.954125 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:50.954046    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1c405568-cb88-4485-ae2c-3614cc31ced6-trusted-ca" pod "1c405568-cb88-4485-ae2c-3614cc31ced6" (UID: "1c405568-cb88-4485-ae2c-3614cc31ced6"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 12:57:50.955828 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:50.955800    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1c405568-cb88-4485-ae2c-3614cc31ced6-installation-pull-secrets" pod "1c405568-cb88-4485-ae2c-3614cc31ced6" (UID: "1c405568-cb88-4485-ae2c-3614cc31ced6"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 12:57:50.955939 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:50.955898    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1c405568-cb88-4485-ae2c-3614cc31ced6-image-registry-private-configuration" pod "1c405568-cb88-4485-ae2c-3614cc31ced6" (UID: "1c405568-cb88-4485-ae2c-3614cc31ced6"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 12:57:50.955939 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:50.955913    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1c405568-cb88-4485-ae2c-3614cc31ced6-kube-api-access-54kft" pod "1c405568-cb88-4485-ae2c-3614cc31ced6" (UID: "1c405568-cb88-4485-ae2c-3614cc31ced6"). InnerVolumeSpecName "kube-api-access-54kft". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 12:57:50.956051 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:50.955946    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1c405568-cb88-4485-ae2c-3614cc31ced6-bound-sa-token" pod "1c405568-cb88-4485-ae2c-3614cc31ced6" (UID: "1c405568-cb88-4485-ae2c-3614cc31ced6"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 12:57:50.956051 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:50.956000    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1c405568-cb88-4485-ae2c-3614cc31ced6-registry-tls" pod "1c405568-cb88-4485-ae2c-3614cc31ced6" (UID: "1c405568-cb88-4485-ae2c-3614cc31ced6"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 12:57:50.960575 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:50.960532    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1c405568-cb88-4485-ae2c-3614cc31ced6-ca-trust-extracted" pod "1c405568-cb88-4485-ae2c-3614cc31ced6" (UID: "1c405568-cb88-4485-ae2c-3614cc31ced6"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 12:57:51.054073 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:51.054044    4943 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-54kft\" (UniqueName: \"kubernetes.io/projected/1c405568-cb88-4485-ae2c-3614cc31ced6-kube-api-access-54kft\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 12:57:51.054073 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:51.054071    4943 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/1c405568-cb88-4485-ae2c-3614cc31ced6-image-registry-private-configuration\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 12:57:51.054225 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:51.054083    4943 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/1c405568-cb88-4485-ae2c-3614cc31ced6-installation-pull-secrets\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 12:57:51.054225 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:51.054092    4943 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1c405568-cb88-4485-ae2c-3614cc31ced6-bound-sa-token\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 12:57:51.054225 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:51.054101    4943 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/1c405568-cb88-4485-ae2c-3614cc31ced6-registry-certificates\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 12:57:51.054225 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:51.054110    4943 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1c405568-cb88-4485-ae2c-3614cc31ced6-registry-tls\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 12:57:51.054225 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:51.054119    4943 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/1c405568-cb88-4485-ae2c-3614cc31ced6-trusted-ca\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 12:57:51.054225 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:51.054127    4943 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/1c405568-cb88-4485-ae2c-3614cc31ced6-ca-trust-extracted\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 12:57:51.633367 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:51.633328    4943 generic.go:356] "Generic (PLEG): container finished" podID="1c405568-cb88-4485-ae2c-3614cc31ced6" containerID="30a1965859c9d662e60abab43d07e297b337a36c46e94386a657899950cf843b" exitCode=0
Oct 30 12:57:51.633854 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:51.633395    4943 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-77c6ff895d-vw6c6"
Oct 30 12:57:51.633854 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:51.633402    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-77c6ff895d-vw6c6" event={"ID":"1c405568-cb88-4485-ae2c-3614cc31ced6","Type":"ContainerDied","Data":"30a1965859c9d662e60abab43d07e297b337a36c46e94386a657899950cf843b"}
Oct 30 12:57:51.633854 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:51.633528    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-77c6ff895d-vw6c6" event={"ID":"1c405568-cb88-4485-ae2c-3614cc31ced6","Type":"ContainerDied","Data":"9d20cc83651a4a89afd1bcf30d72cadacbd043f7b11d1b93f276293ce0789b9f"}
Oct 30 12:57:51.633854 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:51.633547    4943 scope.go:122] "RemoveContainer" containerID="30a1965859c9d662e60abab43d07e297b337a36c46e94386a657899950cf843b"
Oct 30 12:57:51.642062 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:51.642046    4943 scope.go:122] "RemoveContainer" containerID="30a1965859c9d662e60abab43d07e297b337a36c46e94386a657899950cf843b"
Oct 30 12:57:51.642352 ip-10-0-142-84 kubenswrapper[4943]: E1030 12:57:51.642329    4943 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"30a1965859c9d662e60abab43d07e297b337a36c46e94386a657899950cf843b\": container with ID starting with 30a1965859c9d662e60abab43d07e297b337a36c46e94386a657899950cf843b not found: ID does not exist" containerID="30a1965859c9d662e60abab43d07e297b337a36c46e94386a657899950cf843b"
Oct 30 12:57:51.642433 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:51.642358    4943 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"30a1965859c9d662e60abab43d07e297b337a36c46e94386a657899950cf843b"} err="failed to get container status \"30a1965859c9d662e60abab43d07e297b337a36c46e94386a657899950cf843b\": rpc error: code = NotFound desc = could not find container \"30a1965859c9d662e60abab43d07e297b337a36c46e94386a657899950cf843b\": container with ID starting with 30a1965859c9d662e60abab43d07e297b337a36c46e94386a657899950cf843b not found: ID does not exist"
Oct 30 12:57:51.703646 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:51.703611    4943 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-77c6ff895d-vw6c6"]
Oct 30 12:57:51.717166 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:51.717137    4943 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-77c6ff895d-vw6c6"]
Oct 30 12:57:52.348456 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:57:52.348421    4943 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1c405568-cb88-4485-ae2c-3614cc31ced6" path="/var/lib/kubelet/pods/1c405568-cb88-4485-ae2c-3614cc31ced6/volumes"
Oct 30 12:58:05.953479 ip-10-0-142-84 kubenswrapper[4943]: E1030 12:58:05.953440    4943 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-knqz6" containerName="extractor"
Oct 30 12:58:14.152942 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:58:14.152896    4943 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-image-registry/image-registry-69c87564f5-n6wjk" podUID="b86e8e45-3c2d-46b6-9ade-d390b56e4a75" containerName="registry" containerID="cri-o://960327b94c783e263c21e3c5d9ae67f87f458a09caab1213a12b9d9bbf238b0d" gracePeriod=30
Oct 30 12:58:14.386973 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:58:14.386949    4943 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-69c87564f5-n6wjk"
Oct 30 12:58:14.542824 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:58:14.542798    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b86e8e45-3c2d-46b6-9ade-d390b56e4a75-registry-certificates\" (UniqueName: \"kubernetes.io/configmap/b86e8e45-3c2d-46b6-9ade-d390b56e4a75-registry-certificates\") pod \"b86e8e45-3c2d-46b6-9ade-d390b56e4a75\" (UID: \"b86e8e45-3c2d-46b6-9ade-d390b56e4a75\") "
Oct 30 12:58:14.542937 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:58:14.542840    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b86e8e45-3c2d-46b6-9ade-d390b56e4a75-kube-api-access-l8pqr\" (UniqueName: \"kubernetes.io/projected/b86e8e45-3c2d-46b6-9ade-d390b56e4a75-kube-api-access-l8pqr\") pod \"b86e8e45-3c2d-46b6-9ade-d390b56e4a75\" (UID: \"b86e8e45-3c2d-46b6-9ade-d390b56e4a75\") "
Oct 30 12:58:14.542937 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:58:14.542876    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b86e8e45-3c2d-46b6-9ade-d390b56e4a75-registry-tls\" (UniqueName: \"kubernetes.io/projected/b86e8e45-3c2d-46b6-9ade-d390b56e4a75-registry-tls\") pod \"b86e8e45-3c2d-46b6-9ade-d390b56e4a75\" (UID: \"b86e8e45-3c2d-46b6-9ade-d390b56e4a75\") "
Oct 30 12:58:14.542937 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:58:14.542921    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b86e8e45-3c2d-46b6-9ade-d390b56e4a75-ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/b86e8e45-3c2d-46b6-9ade-d390b56e4a75-ca-trust-extracted\") pod \"b86e8e45-3c2d-46b6-9ade-d390b56e4a75\" (UID: \"b86e8e45-3c2d-46b6-9ade-d390b56e4a75\") "
Oct 30 12:58:14.543096 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:58:14.542973    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/b86e8e45-3c2d-46b6-9ade-d390b56e4a75-image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/b86e8e45-3c2d-46b6-9ade-d390b56e4a75-image-registry-private-configuration\") pod \"b86e8e45-3c2d-46b6-9ade-d390b56e4a75\" (UID: \"b86e8e45-3c2d-46b6-9ade-d390b56e4a75\") "
Oct 30 12:58:14.543096 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:58:14.543011    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b86e8e45-3c2d-46b6-9ade-d390b56e4a75-trusted-ca\" (UniqueName: \"kubernetes.io/configmap/b86e8e45-3c2d-46b6-9ade-d390b56e4a75-trusted-ca\") pod \"b86e8e45-3c2d-46b6-9ade-d390b56e4a75\" (UID: \"b86e8e45-3c2d-46b6-9ade-d390b56e4a75\") "
Oct 30 12:58:14.543194 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:58:14.543143    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/b86e8e45-3c2d-46b6-9ade-d390b56e4a75-installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/b86e8e45-3c2d-46b6-9ade-d390b56e4a75-installation-pull-secrets\") pod \"b86e8e45-3c2d-46b6-9ade-d390b56e4a75\" (UID: \"b86e8e45-3c2d-46b6-9ade-d390b56e4a75\") "
Oct 30 12:58:14.543247 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:58:14.543193    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b86e8e45-3c2d-46b6-9ade-d390b56e4a75-bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b86e8e45-3c2d-46b6-9ade-d390b56e4a75-bound-sa-token\") pod \"b86e8e45-3c2d-46b6-9ade-d390b56e4a75\" (UID: \"b86e8e45-3c2d-46b6-9ade-d390b56e4a75\") "
Oct 30 12:58:14.543408 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:58:14.543374    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b86e8e45-3c2d-46b6-9ade-d390b56e4a75-registry-certificates" pod "b86e8e45-3c2d-46b6-9ade-d390b56e4a75" (UID: "b86e8e45-3c2d-46b6-9ade-d390b56e4a75"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 12:58:14.543488 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:58:14.543471    4943 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/b86e8e45-3c2d-46b6-9ade-d390b56e4a75-registry-certificates\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 12:58:14.543549 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:58:14.543470    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b86e8e45-3c2d-46b6-9ade-d390b56e4a75-trusted-ca" pod "b86e8e45-3c2d-46b6-9ade-d390b56e4a75" (UID: "b86e8e45-3c2d-46b6-9ade-d390b56e4a75"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 12:58:14.545747 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:58:14.545723    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b86e8e45-3c2d-46b6-9ade-d390b56e4a75-installation-pull-secrets" pod "b86e8e45-3c2d-46b6-9ade-d390b56e4a75" (UID: "b86e8e45-3c2d-46b6-9ade-d390b56e4a75"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 12:58:14.545747 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:58:14.545734    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b86e8e45-3c2d-46b6-9ade-d390b56e4a75-kube-api-access-l8pqr" pod "b86e8e45-3c2d-46b6-9ade-d390b56e4a75" (UID: "b86e8e45-3c2d-46b6-9ade-d390b56e4a75"). InnerVolumeSpecName "kube-api-access-l8pqr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 12:58:14.545886 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:58:14.545742    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b86e8e45-3c2d-46b6-9ade-d390b56e4a75-registry-tls" pod "b86e8e45-3c2d-46b6-9ade-d390b56e4a75" (UID: "b86e8e45-3c2d-46b6-9ade-d390b56e4a75"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 12:58:14.545886 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:58:14.545726    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b86e8e45-3c2d-46b6-9ade-d390b56e4a75-image-registry-private-configuration" pod "b86e8e45-3c2d-46b6-9ade-d390b56e4a75" (UID: "b86e8e45-3c2d-46b6-9ade-d390b56e4a75"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 12:58:14.545886 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:58:14.545726    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b86e8e45-3c2d-46b6-9ade-d390b56e4a75-bound-sa-token" pod "b86e8e45-3c2d-46b6-9ade-d390b56e4a75" (UID: "b86e8e45-3c2d-46b6-9ade-d390b56e4a75"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 12:58:14.549488 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:58:14.549461    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b86e8e45-3c2d-46b6-9ade-d390b56e4a75-ca-trust-extracted" pod "b86e8e45-3c2d-46b6-9ade-d390b56e4a75" (UID: "b86e8e45-3c2d-46b6-9ade-d390b56e4a75"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 12:58:14.644359 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:58:14.644338    4943 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/b86e8e45-3c2d-46b6-9ade-d390b56e4a75-ca-trust-extracted\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 12:58:14.644359 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:58:14.644359    4943 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/b86e8e45-3c2d-46b6-9ade-d390b56e4a75-image-registry-private-configuration\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 12:58:14.644518 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:58:14.644370    4943 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/b86e8e45-3c2d-46b6-9ade-d390b56e4a75-trusted-ca\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 12:58:14.644518 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:58:14.644380    4943 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/b86e8e45-3c2d-46b6-9ade-d390b56e4a75-installation-pull-secrets\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 12:58:14.644518 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:58:14.644390    4943 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b86e8e45-3c2d-46b6-9ade-d390b56e4a75-bound-sa-token\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 12:58:14.644518 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:58:14.644398    4943 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l8pqr\" (UniqueName: \"kubernetes.io/projected/b86e8e45-3c2d-46b6-9ade-d390b56e4a75-kube-api-access-l8pqr\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 12:58:14.644518 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:58:14.644407    4943 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b86e8e45-3c2d-46b6-9ade-d390b56e4a75-registry-tls\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 12:58:14.720810 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:58:14.720781    4943 generic.go:356] "Generic (PLEG): container finished" podID="bd9e55af-64ad-4ffc-92c3-53b7bf654322" containerID="2866a14b88005f3ca49c5f98bfd7f492682aea58311f99d2d422c491262bf6dc" exitCode=0
Oct 30 12:58:14.720954 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:58:14.720857    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-7c4986d449-7cfvt" event={"ID":"bd9e55af-64ad-4ffc-92c3-53b7bf654322","Type":"ContainerDied","Data":"2866a14b88005f3ca49c5f98bfd7f492682aea58311f99d2d422c491262bf6dc"}
Oct 30 12:58:14.720954 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:58:14.720919    4943 scope.go:122] "RemoveContainer" containerID="e77daa452696e3d2aec1881194c78a39904af7eed8db1d49200f93af317a9960"
Oct 30 12:58:14.721245 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:58:14.721222    4943 scope.go:122] "RemoveContainer" containerID="2866a14b88005f3ca49c5f98bfd7f492682aea58311f99d2d422c491262bf6dc"
Oct 30 12:58:14.721500 ip-10-0-142-84 kubenswrapper[4943]: E1030 12:58:14.721480    4943 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"insights-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=insights-operator pod=insights-operator-7c4986d449-7cfvt_openshift-insights(bd9e55af-64ad-4ffc-92c3-53b7bf654322)\"" pod="openshift-insights/insights-operator-7c4986d449-7cfvt" podUID="bd9e55af-64ad-4ffc-92c3-53b7bf654322"
Oct 30 12:58:14.722168 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:58:14.722148    4943 generic.go:356] "Generic (PLEG): container finished" podID="b86e8e45-3c2d-46b6-9ade-d390b56e4a75" containerID="960327b94c783e263c21e3c5d9ae67f87f458a09caab1213a12b9d9bbf238b0d" exitCode=0
Oct 30 12:58:14.722253 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:58:14.722187    4943 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-69c87564f5-n6wjk"
Oct 30 12:58:14.722253 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:58:14.722202    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-69c87564f5-n6wjk" event={"ID":"b86e8e45-3c2d-46b6-9ade-d390b56e4a75","Type":"ContainerDied","Data":"960327b94c783e263c21e3c5d9ae67f87f458a09caab1213a12b9d9bbf238b0d"}
Oct 30 12:58:14.722253 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:58:14.722228    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-69c87564f5-n6wjk" event={"ID":"b86e8e45-3c2d-46b6-9ade-d390b56e4a75","Type":"ContainerDied","Data":"89790d76787d33e93317b53de75aba614499ee43ae7d9b7aac5e8f6c0f9723f6"}
Oct 30 12:58:14.733600 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:58:14.733581    4943 scope.go:122] "RemoveContainer" containerID="960327b94c783e263c21e3c5d9ae67f87f458a09caab1213a12b9d9bbf238b0d"
Oct 30 12:58:14.743091 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:58:14.743075    4943 scope.go:122] "RemoveContainer" containerID="960327b94c783e263c21e3c5d9ae67f87f458a09caab1213a12b9d9bbf238b0d"
Oct 30 12:58:14.743359 ip-10-0-142-84 kubenswrapper[4943]: E1030 12:58:14.743341    4943 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"960327b94c783e263c21e3c5d9ae67f87f458a09caab1213a12b9d9bbf238b0d\": container with ID starting with 960327b94c783e263c21e3c5d9ae67f87f458a09caab1213a12b9d9bbf238b0d not found: ID does not exist" containerID="960327b94c783e263c21e3c5d9ae67f87f458a09caab1213a12b9d9bbf238b0d"
Oct 30 12:58:14.743431 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:58:14.743369    4943 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"960327b94c783e263c21e3c5d9ae67f87f458a09caab1213a12b9d9bbf238b0d"} err="failed to get container status \"960327b94c783e263c21e3c5d9ae67f87f458a09caab1213a12b9d9bbf238b0d\": rpc error: code = NotFound desc = could not find container \"960327b94c783e263c21e3c5d9ae67f87f458a09caab1213a12b9d9bbf238b0d\": container with ID starting with 960327b94c783e263c21e3c5d9ae67f87f458a09caab1213a12b9d9bbf238b0d not found: ID does not exist"
Oct 30 12:58:14.809667 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:58:14.809598    4943 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-69c87564f5-n6wjk"]
Oct 30 12:58:14.822328 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:58:14.822307    4943 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-69c87564f5-n6wjk"]
Oct 30 12:58:16.347565 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:58:16.347525    4943 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b86e8e45-3c2d-46b6-9ade-d390b56e4a75" path="/var/lib/kubelet/pods/b86e8e45-3c2d-46b6-9ade-d390b56e4a75/volumes"
Oct 30 12:58:19.746681 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:58:19.746650    4943 generic.go:356] "Generic (PLEG): container finished" podID="27c55f84-4e9a-44de-94ea-02258d6bc747" containerID="be9e4b58df52832bf9f3009bd3440bc1f53c0e0245716f42bfe0301d5b3e2141" exitCode=0
Oct 30 12:58:19.747050 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:58:19.746700    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-h42v2" event={"ID":"27c55f84-4e9a-44de-94ea-02258d6bc747","Type":"ContainerDied","Data":"be9e4b58df52832bf9f3009bd3440bc1f53c0e0245716f42bfe0301d5b3e2141"}
Oct 30 12:58:19.747050 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:58:19.746951    4943 scope.go:122] "RemoveContainer" containerID="be9e4b58df52832bf9f3009bd3440bc1f53c0e0245716f42bfe0301d5b3e2141"
Oct 30 12:58:20.751735 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:58:20.751700    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-h42v2" event={"ID":"27c55f84-4e9a-44de-94ea-02258d6bc747","Type":"ContainerStarted","Data":"1e25086c492a09290f42e2423ba9ce649f1c9952c7197908868e8c62223ac4e5"}
Oct 30 12:58:26.342920 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:58:26.342887    4943 scope.go:122] "RemoveContainer" containerID="2866a14b88005f3ca49c5f98bfd7f492682aea58311f99d2d422c491262bf6dc"
Oct 30 12:58:26.775146 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:58:26.775113    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-7c4986d449-7cfvt" event={"ID":"bd9e55af-64ad-4ffc-92c3-53b7bf654322","Type":"ContainerStarted","Data":"352098fa9e97e006043c2d78d349f1682e007facdae98ed12c5eb60e2017bd71"}
Oct 30 12:58:29.787169 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:58:29.787140    4943 generic.go:356] "Generic (PLEG): container finished" podID="85f4c399-0089-4399-86e2-069834f89286" containerID="095c72e4ebb12fd5c0f400652dd933d4d6ee0b1e134430ae1bec9fde18a6aa98" exitCode=0
Oct 30 12:58:29.787513 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:58:29.787199    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-plmkt" event={"ID":"85f4c399-0089-4399-86e2-069834f89286","Type":"ContainerDied","Data":"095c72e4ebb12fd5c0f400652dd933d4d6ee0b1e134430ae1bec9fde18a6aa98"}
Oct 30 12:58:29.787513 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:58:29.787495    4943 scope.go:122] "RemoveContainer" containerID="095c72e4ebb12fd5c0f400652dd933d4d6ee0b1e134430ae1bec9fde18a6aa98"
Oct 30 12:58:30.793018 ip-10-0-142-84 kubenswrapper[4943]: I1030 12:58:30.792984    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-plmkt" event={"ID":"85f4c399-0089-4399-86e2-069834f89286","Type":"ContainerStarted","Data":"585dde2f1651da82bde5569c1097fdcafc424974eeb9e10617d55d79c7161bce"}
Oct 30 12:58:35.953919 ip-10-0-142-84 kubenswrapper[4943]: E1030 12:58:35.953879    4943 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-knqz6" containerName="extractor"
Oct 30 12:59:05.954317 ip-10-0-142-84 kubenswrapper[4943]: E1030 12:59:05.954277    4943 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-knqz6" containerName="extractor"
Oct 30 12:59:35.954243 ip-10-0-142-84 kubenswrapper[4943]: E1030 12:59:35.954161    4943 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-knqz6" containerName="extractor"
Oct 30 13:00:05.954470 ip-10-0-142-84 kubenswrapper[4943]: E1030 13:00:05.954431    4943 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-knqz6" containerName="extractor"
Oct 30 13:00:35.953867 ip-10-0-142-84 kubenswrapper[4943]: E1030 13:00:35.953824    4943 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-knqz6" containerName="extractor"
Oct 30 13:01:05.953195 ip-10-0-142-84 kubenswrapper[4943]: E1030 13:01:05.953153    4943 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-knqz6" containerName="extractor"
Oct 30 13:01:35.953529 ip-10-0-142-84 kubenswrapper[4943]: E1030 13:01:35.953490    4943 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-knqz6" containerName="extractor"
Oct 30 13:01:50.273117 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:01:50.273081    4943 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651ckz28g"]
Oct 30 13:01:50.273543 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:01:50.273410    4943 cpu_manager.go:414] "containerMap: removing container" podUID="1c405568-cb88-4485-ae2c-3614cc31ced6" containerName="registry"
Oct 30 13:01:50.273543 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:01:50.273421    4943 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1c405568-cb88-4485-ae2c-3614cc31ced6" containerName="registry"
Oct 30 13:01:50.273543 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:01:50.273434    4943 cpu_manager.go:414] "containerMap: removing container" podUID="b86e8e45-3c2d-46b6-9ade-d390b56e4a75" containerName="registry"
Oct 30 13:01:50.273543 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:01:50.273438    4943 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b86e8e45-3c2d-46b6-9ade-d390b56e4a75" containerName="registry"
Oct 30 13:01:50.273543 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:01:50.273492    4943 memory_manager.go:365] "RemoveStaleState removing state" podUID="1c405568-cb88-4485-ae2c-3614cc31ced6" containerName="registry"
Oct 30 13:01:50.273543 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:01:50.273501    4943 memory_manager.go:365] "RemoveStaleState removing state" podUID="b86e8e45-3c2d-46b6-9ade-d390b56e4a75" containerName="registry"
Oct 30 13:01:50.276540 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:01:50.276522    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651ckz28g"
Oct 30 13:01:50.308756 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:01:50.308719    4943 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Oct 30 13:01:50.308918 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:01:50.308794    4943 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Oct 30 13:01:50.308918 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:01:50.308810    4943 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"c7537007944577f64df34960ace3c69136c416-139ec370-dockercfg-q5svf\""
Oct 30 13:01:50.316150 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:01:50.316125    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651ckz28g"]
Oct 30 13:01:50.367108 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:01:50.367075    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a5a1d5a7-18e3-4e06-94eb-4dca54aa5dd3-bundle\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651ckz28g\" (UID: \"a5a1d5a7-18e3-4e06-94eb-4dca54aa5dd3\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651ckz28g"
Oct 30 13:01:50.367298 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:01:50.367118    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a5a1d5a7-18e3-4e06-94eb-4dca54aa5dd3-util\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651ckz28g\" (UID: \"a5a1d5a7-18e3-4e06-94eb-4dca54aa5dd3\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651ckz28g"
Oct 30 13:01:50.367298 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:01:50.367226    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x6twt\" (UniqueName: \"kubernetes.io/projected/a5a1d5a7-18e3-4e06-94eb-4dca54aa5dd3-kube-api-access-x6twt\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651ckz28g\" (UID: \"a5a1d5a7-18e3-4e06-94eb-4dca54aa5dd3\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651ckz28g"
Oct 30 13:01:50.467698 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:01:50.467658    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a5a1d5a7-18e3-4e06-94eb-4dca54aa5dd3-util\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651ckz28g\" (UID: \"a5a1d5a7-18e3-4e06-94eb-4dca54aa5dd3\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651ckz28g"
Oct 30 13:01:50.467892 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:01:50.467742    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x6twt\" (UniqueName: \"kubernetes.io/projected/a5a1d5a7-18e3-4e06-94eb-4dca54aa5dd3-kube-api-access-x6twt\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651ckz28g\" (UID: \"a5a1d5a7-18e3-4e06-94eb-4dca54aa5dd3\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651ckz28g"
Oct 30 13:01:50.467892 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:01:50.467824    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a5a1d5a7-18e3-4e06-94eb-4dca54aa5dd3-bundle\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651ckz28g\" (UID: \"a5a1d5a7-18e3-4e06-94eb-4dca54aa5dd3\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651ckz28g"
Oct 30 13:01:50.468109 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:01:50.468084    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a5a1d5a7-18e3-4e06-94eb-4dca54aa5dd3-util\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651ckz28g\" (UID: \"a5a1d5a7-18e3-4e06-94eb-4dca54aa5dd3\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651ckz28g"
Oct 30 13:01:50.468179 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:01:50.468140    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a5a1d5a7-18e3-4e06-94eb-4dca54aa5dd3-bundle\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651ckz28g\" (UID: \"a5a1d5a7-18e3-4e06-94eb-4dca54aa5dd3\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651ckz28g"
Oct 30 13:01:50.493626 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:01:50.493596    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x6twt\" (UniqueName: \"kubernetes.io/projected/a5a1d5a7-18e3-4e06-94eb-4dca54aa5dd3-kube-api-access-x6twt\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651ckz28g\" (UID: \"a5a1d5a7-18e3-4e06-94eb-4dca54aa5dd3\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651ckz28g"
Oct 30 13:01:50.591314 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:01:50.591225    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651ckz28g"
Oct 30 13:01:50.891172 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:01:50.891099    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651ckz28g"]
Oct 30 13:01:50.894560 ip-10-0-142-84 kubenswrapper[4943]: W1030 13:01:50.894526    4943 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda5a1d5a7_18e3_4e06_94eb_4dca54aa5dd3.slice/crio-d50a45a9149d47487f5f1c5834036cb0a39d5cc3c969f4cf647cb90b09cf23ac WatchSource:0}: Error finding container d50a45a9149d47487f5f1c5834036cb0a39d5cc3c969f4cf647cb90b09cf23ac: Status 404 returned error can't find the container with id d50a45a9149d47487f5f1c5834036cb0a39d5cc3c969f4cf647cb90b09cf23ac
Oct 30 13:01:51.475371 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:01:51.475326    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651ckz28g" event={"ID":"a5a1d5a7-18e3-4e06-94eb-4dca54aa5dd3","Type":"ContainerStarted","Data":"d50a45a9149d47487f5f1c5834036cb0a39d5cc3c969f4cf647cb90b09cf23ac"}
Oct 30 13:01:56.496308 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:01:56.496255    4943 generic.go:356] "Generic (PLEG): container finished" podID="a5a1d5a7-18e3-4e06-94eb-4dca54aa5dd3" containerID="5625626d020db8fbdc4623c9b56b99105cc3dc9feb5c8034f899602f68a60b0c" exitCode=0
Oct 30 13:01:56.496308 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:01:56.496310    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651ckz28g" event={"ID":"a5a1d5a7-18e3-4e06-94eb-4dca54aa5dd3","Type":"ContainerDied","Data":"5625626d020db8fbdc4623c9b56b99105cc3dc9feb5c8034f899602f68a60b0c"}
Oct 30 13:01:59.511236 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:01:59.511194    4943 generic.go:356] "Generic (PLEG): container finished" podID="a5a1d5a7-18e3-4e06-94eb-4dca54aa5dd3" containerID="d6cc05d612482eff6e99cad00cc0e4c452f8d7b97ba7e6f083d64c8f4c06767c" exitCode=0
Oct 30 13:01:59.511660 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:01:59.511292    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651ckz28g" event={"ID":"a5a1d5a7-18e3-4e06-94eb-4dca54aa5dd3","Type":"ContainerDied","Data":"d6cc05d612482eff6e99cad00cc0e4c452f8d7b97ba7e6f083d64c8f4c06767c"}
Oct 30 13:02:04.358013 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:02:04.357979    4943 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-8fqfr_d8930ee7-0ae0-48b5-8ce3-cb8820355cef/console-operator/2.log"
Oct 30 13:02:04.359053 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:02:04.358904    4943 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-8fqfr_d8930ee7-0ae0-48b5-8ce3-cb8820355cef/console-operator/2.log"
Oct 30 13:02:05.535889 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:02:05.535855    4943 generic.go:356] "Generic (PLEG): container finished" podID="a5a1d5a7-18e3-4e06-94eb-4dca54aa5dd3" containerID="89afc0a15953c426e4e47450ec4fa8257ede4c3e1927db15db2e74cd8d8ee572" exitCode=0
Oct 30 13:02:05.536314 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:02:05.535941    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651ckz28g" event={"ID":"a5a1d5a7-18e3-4e06-94eb-4dca54aa5dd3","Type":"ContainerDied","Data":"89afc0a15953c426e4e47450ec4fa8257ede4c3e1927db15db2e74cd8d8ee572"}
Oct 30 13:02:05.957634 ip-10-0-142-84 kubenswrapper[4943]: E1030 13:02:05.957593    4943 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-knqz6" containerName="extractor"
Oct 30 13:02:06.664982 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:02:06.664957    4943 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651ckz28g"
Oct 30 13:02:06.808096 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:02:06.808004    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a5a1d5a7-18e3-4e06-94eb-4dca54aa5dd3-kube-api-access-x6twt\" (UniqueName: \"kubernetes.io/projected/a5a1d5a7-18e3-4e06-94eb-4dca54aa5dd3-kube-api-access-x6twt\") pod \"a5a1d5a7-18e3-4e06-94eb-4dca54aa5dd3\" (UID: \"a5a1d5a7-18e3-4e06-94eb-4dca54aa5dd3\") "
Oct 30 13:02:06.808096 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:02:06.808053    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a5a1d5a7-18e3-4e06-94eb-4dca54aa5dd3-bundle\" (UniqueName: \"kubernetes.io/empty-dir/a5a1d5a7-18e3-4e06-94eb-4dca54aa5dd3-bundle\") pod \"a5a1d5a7-18e3-4e06-94eb-4dca54aa5dd3\" (UID: \"a5a1d5a7-18e3-4e06-94eb-4dca54aa5dd3\") "
Oct 30 13:02:06.808096 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:02:06.808080    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a5a1d5a7-18e3-4e06-94eb-4dca54aa5dd3-util\" (UniqueName: \"kubernetes.io/empty-dir/a5a1d5a7-18e3-4e06-94eb-4dca54aa5dd3-util\") pod \"a5a1d5a7-18e3-4e06-94eb-4dca54aa5dd3\" (UID: \"a5a1d5a7-18e3-4e06-94eb-4dca54aa5dd3\") "
Oct 30 13:02:06.808703 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:02:06.808673    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a5a1d5a7-18e3-4e06-94eb-4dca54aa5dd3-bundle" pod "a5a1d5a7-18e3-4e06-94eb-4dca54aa5dd3" (UID: "a5a1d5a7-18e3-4e06-94eb-4dca54aa5dd3"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:02:06.810362 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:02:06.810337    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a5a1d5a7-18e3-4e06-94eb-4dca54aa5dd3-kube-api-access-x6twt" pod "a5a1d5a7-18e3-4e06-94eb-4dca54aa5dd3" (UID: "a5a1d5a7-18e3-4e06-94eb-4dca54aa5dd3"). InnerVolumeSpecName "kube-api-access-x6twt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:02:06.812377 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:02:06.812346    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a5a1d5a7-18e3-4e06-94eb-4dca54aa5dd3-util" pod "a5a1d5a7-18e3-4e06-94eb-4dca54aa5dd3" (UID: "a5a1d5a7-18e3-4e06-94eb-4dca54aa5dd3"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:02:06.908716 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:02:06.908670    4943 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x6twt\" (UniqueName: \"kubernetes.io/projected/a5a1d5a7-18e3-4e06-94eb-4dca54aa5dd3-kube-api-access-x6twt\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:02:06.908716 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:02:06.908711    4943 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a5a1d5a7-18e3-4e06-94eb-4dca54aa5dd3-bundle\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:02:06.908716 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:02:06.908725    4943 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a5a1d5a7-18e3-4e06-94eb-4dca54aa5dd3-util\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:02:07.545204 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:02:07.545169    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651ckz28g" event={"ID":"a5a1d5a7-18e3-4e06-94eb-4dca54aa5dd3","Type":"ContainerDied","Data":"d50a45a9149d47487f5f1c5834036cb0a39d5cc3c969f4cf647cb90b09cf23ac"}
Oct 30 13:02:07.545204 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:02:07.545206    4943 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651ckz28g"
Oct 30 13:02:07.545510 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:02:07.545218    4943 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d50a45a9149d47487f5f1c5834036cb0a39d5cc3c969f4cf647cb90b09cf23ac"
Oct 30 13:02:13.842449 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:02:13.842413    4943 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-89fdbf7dc-2v98j"]
Oct 30 13:02:13.842928 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:02:13.842815    4943 cpu_manager.go:414] "containerMap: removing container" podUID="a5a1d5a7-18e3-4e06-94eb-4dca54aa5dd3" containerName="pull"
Oct 30 13:02:13.842928 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:02:13.842832    4943 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a5a1d5a7-18e3-4e06-94eb-4dca54aa5dd3" containerName="pull"
Oct 30 13:02:13.842928 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:02:13.842884    4943 cpu_manager.go:414] "containerMap: removing container" podUID="a5a1d5a7-18e3-4e06-94eb-4dca54aa5dd3" containerName="extract"
Oct 30 13:02:13.842928 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:02:13.842893    4943 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a5a1d5a7-18e3-4e06-94eb-4dca54aa5dd3" containerName="extract"
Oct 30 13:02:13.842928 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:02:13.842907    4943 cpu_manager.go:414] "containerMap: removing container" podUID="a5a1d5a7-18e3-4e06-94eb-4dca54aa5dd3" containerName="util"
Oct 30 13:02:13.842928 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:02:13.842914    4943 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a5a1d5a7-18e3-4e06-94eb-4dca54aa5dd3" containerName="util"
Oct 30 13:02:13.843214 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:02:13.843011    4943 memory_manager.go:365] "RemoveStaleState removing state" podUID="a5a1d5a7-18e3-4e06-94eb-4dca54aa5dd3" containerName="extract"
Oct 30 13:02:13.845101 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:02:13.845079    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-89fdbf7dc-2v98j"
Oct 30 13:02:13.866058 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:02:13.866034    4943 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-g67dt\""
Oct 30 13:02:13.879652 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:02:13.879634    4943 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\""
Oct 30 13:02:13.879753 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:02:13.879701    4943 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\""
Oct 30 13:02:13.880745 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:02:13.880717    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-89fdbf7dc-2v98j"]
Oct 30 13:02:13.960442 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:02:13.960416    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8cvfq\" (UniqueName: \"kubernetes.io/projected/d3575af9-6775-4bf9-9b18-f49d02b2af08-kube-api-access-8cvfq\") pod \"cert-manager-operator-controller-manager-89fdbf7dc-2v98j\" (UID: \"d3575af9-6775-4bf9-9b18-f49d02b2af08\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-89fdbf7dc-2v98j"
Oct 30 13:02:13.960562 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:02:13.960452    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d3575af9-6775-4bf9-9b18-f49d02b2af08-tmp\") pod \"cert-manager-operator-controller-manager-89fdbf7dc-2v98j\" (UID: \"d3575af9-6775-4bf9-9b18-f49d02b2af08\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-89fdbf7dc-2v98j"
Oct 30 13:02:14.061387 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:02:14.061361    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8cvfq\" (UniqueName: \"kubernetes.io/projected/d3575af9-6775-4bf9-9b18-f49d02b2af08-kube-api-access-8cvfq\") pod \"cert-manager-operator-controller-manager-89fdbf7dc-2v98j\" (UID: \"d3575af9-6775-4bf9-9b18-f49d02b2af08\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-89fdbf7dc-2v98j"
Oct 30 13:02:14.061543 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:02:14.061394    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d3575af9-6775-4bf9-9b18-f49d02b2af08-tmp\") pod \"cert-manager-operator-controller-manager-89fdbf7dc-2v98j\" (UID: \"d3575af9-6775-4bf9-9b18-f49d02b2af08\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-89fdbf7dc-2v98j"
Oct 30 13:02:14.061783 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:02:14.061765    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d3575af9-6775-4bf9-9b18-f49d02b2af08-tmp\") pod \"cert-manager-operator-controller-manager-89fdbf7dc-2v98j\" (UID: \"d3575af9-6775-4bf9-9b18-f49d02b2af08\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-89fdbf7dc-2v98j"
Oct 30 13:02:14.087221 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:02:14.087199    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8cvfq\" (UniqueName: \"kubernetes.io/projected/d3575af9-6775-4bf9-9b18-f49d02b2af08-kube-api-access-8cvfq\") pod \"cert-manager-operator-controller-manager-89fdbf7dc-2v98j\" (UID: \"d3575af9-6775-4bf9-9b18-f49d02b2af08\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-89fdbf7dc-2v98j"
Oct 30 13:02:14.154745 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:02:14.154691    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-89fdbf7dc-2v98j"
Oct 30 13:02:14.485673 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:02:14.485550    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-89fdbf7dc-2v98j"]
Oct 30 13:02:14.489101 ip-10-0-142-84 kubenswrapper[4943]: W1030 13:02:14.489075    4943 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd3575af9_6775_4bf9_9b18_f49d02b2af08.slice/crio-d64d1de9122f82bc8cbc50255f363c70e819936e0a58ff3d2c615403792cae21 WatchSource:0}: Error finding container d64d1de9122f82bc8cbc50255f363c70e819936e0a58ff3d2c615403792cae21: Status 404 returned error can't find the container with id d64d1de9122f82bc8cbc50255f363c70e819936e0a58ff3d2c615403792cae21
Oct 30 13:02:14.492100 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:02:14.492083    4943 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 13:02:14.573304 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:02:14.573278    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-89fdbf7dc-2v98j" event={"ID":"d3575af9-6775-4bf9-9b18-f49d02b2af08","Type":"ContainerStarted","Data":"d64d1de9122f82bc8cbc50255f363c70e819936e0a58ff3d2c615403792cae21"}
Oct 30 13:02:25.617448 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:02:25.617359    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-89fdbf7dc-2v98j" event={"ID":"d3575af9-6775-4bf9-9b18-f49d02b2af08","Type":"ContainerStarted","Data":"5dec9f27ef566aaade52de99ab198a6d4898db0da052c020ad4440a38c28e05f"}
Oct 30 13:02:25.724852 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:02:25.724791    4943 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-89fdbf7dc-2v98j" podStartSLOduration=1.908418918 podStartE2EDuration="12.724777995s" totalImagesPullingTime="10.816359077s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:02:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:02:25.709602017 +0000 UTC m=+321.497605747" watchObservedRunningTime="2026-10-30 13:02:25.724777995 +0000 UTC m=+321.512781724"
Oct 30 13:02:34.119678 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:02:34.119646    4943 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-5c57bbd4fc-4xmnh"]
Oct 30 13:02:34.124383 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:02:34.124362    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-4xmnh"
Oct 30 13:02:34.144156 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:02:34.144135    4943 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Oct 30 13:02:34.145211 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:02:34.145190    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-5c57bbd4fc-4xmnh"]
Oct 30 13:02:34.157751 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:02:34.157733    4943 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Oct 30 13:02:34.157972 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:02:34.157955    4943 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-xdbpg\""
Oct 30 13:02:34.308651 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:02:34.308619    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d38d159c-a25a-4543-bf52-ca709fafbd69-bound-sa-token\") pod \"cert-manager-cainjector-5c57bbd4fc-4xmnh\" (UID: \"d38d159c-a25a-4543-bf52-ca709fafbd69\") " pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-4xmnh"
Oct 30 13:02:34.308771 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:02:34.308659    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h6zp6\" (UniqueName: \"kubernetes.io/projected/d38d159c-a25a-4543-bf52-ca709fafbd69-kube-api-access-h6zp6\") pod \"cert-manager-cainjector-5c57bbd4fc-4xmnh\" (UID: \"d38d159c-a25a-4543-bf52-ca709fafbd69\") " pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-4xmnh"
Oct 30 13:02:34.409186 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:02:34.409115    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d38d159c-a25a-4543-bf52-ca709fafbd69-bound-sa-token\") pod \"cert-manager-cainjector-5c57bbd4fc-4xmnh\" (UID: \"d38d159c-a25a-4543-bf52-ca709fafbd69\") " pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-4xmnh"
Oct 30 13:02:34.409186 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:02:34.409155    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h6zp6\" (UniqueName: \"kubernetes.io/projected/d38d159c-a25a-4543-bf52-ca709fafbd69-kube-api-access-h6zp6\") pod \"cert-manager-cainjector-5c57bbd4fc-4xmnh\" (UID: \"d38d159c-a25a-4543-bf52-ca709fafbd69\") " pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-4xmnh"
Oct 30 13:02:34.434908 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:02:34.434877    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d38d159c-a25a-4543-bf52-ca709fafbd69-bound-sa-token\") pod \"cert-manager-cainjector-5c57bbd4fc-4xmnh\" (UID: \"d38d159c-a25a-4543-bf52-ca709fafbd69\") " pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-4xmnh"
Oct 30 13:02:34.435014 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:02:34.434971    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h6zp6\" (UniqueName: \"kubernetes.io/projected/d38d159c-a25a-4543-bf52-ca709fafbd69-kube-api-access-h6zp6\") pod \"cert-manager-cainjector-5c57bbd4fc-4xmnh\" (UID: \"d38d159c-a25a-4543-bf52-ca709fafbd69\") " pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-4xmnh"
Oct 30 13:02:34.448742 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:02:34.448718    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-4xmnh"
Oct 30 13:02:34.753589 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:02:34.753564    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-5c57bbd4fc-4xmnh"]
Oct 30 13:02:34.755800 ip-10-0-142-84 kubenswrapper[4943]: W1030 13:02:34.755770    4943 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd38d159c_a25a_4543_bf52_ca709fafbd69.slice/crio-8fc9589238d4e5e66733ed642ea2af9ca832100d8b37de4f62e74c9a215bab28 WatchSource:0}: Error finding container 8fc9589238d4e5e66733ed642ea2af9ca832100d8b37de4f62e74c9a215bab28: Status 404 returned error can't find the container with id 8fc9589238d4e5e66733ed642ea2af9ca832100d8b37de4f62e74c9a215bab28
Oct 30 13:02:35.658416 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:02:35.658379    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-4xmnh" event={"ID":"d38d159c-a25a-4543-bf52-ca709fafbd69","Type":"ContainerStarted","Data":"8fc9589238d4e5e66733ed642ea2af9ca832100d8b37de4f62e74c9a215bab28"}
Oct 30 13:02:35.959853 ip-10-0-142-84 kubenswrapper[4943]: E1030 13:02:35.959814    4943 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-knqz6" containerName="extractor"
Oct 30 13:02:36.615553 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:02:36.615517    4943 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/61b1ff68c7255ee5c5329fe19a5d96293a95788d9734a0a79829e8dc85ljjpk"]
Oct 30 13:02:36.628677 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:02:36.628649    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/61b1ff68c7255ee5c5329fe19a5d96293a95788d9734a0a79829e8dc85ljjpk"
Oct 30 13:02:36.641706 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:02:36.641682    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/61b1ff68c7255ee5c5329fe19a5d96293a95788d9734a0a79829e8dc85ljjpk"]
Oct 30 13:02:36.653689 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:02:36.653632    4943 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Oct 30 13:02:36.653855 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:02:36.653804    4943 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Oct 30 13:02:36.654543 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:02:36.654520    4943 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"61b1ff68c7255ee5c5329fe19a5d96293a9578-ce91a85d-dockercfg-2l2kr\""
Oct 30 13:02:36.728329 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:02:36.728292    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ba8803d2-ea4e-4183-ab00-62f7b1fd85df-bundle\") pod \"61b1ff68c7255ee5c5329fe19a5d96293a95788d9734a0a79829e8dc85ljjpk\" (UID: \"ba8803d2-ea4e-4183-ab00-62f7b1fd85df\") " pod="openshift-marketplace/61b1ff68c7255ee5c5329fe19a5d96293a95788d9734a0a79829e8dc85ljjpk"
Oct 30 13:02:36.728761 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:02:36.728425    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tv8w4\" (UniqueName: \"kubernetes.io/projected/ba8803d2-ea4e-4183-ab00-62f7b1fd85df-kube-api-access-tv8w4\") pod \"61b1ff68c7255ee5c5329fe19a5d96293a95788d9734a0a79829e8dc85ljjpk\" (UID: \"ba8803d2-ea4e-4183-ab00-62f7b1fd85df\") " pod="openshift-marketplace/61b1ff68c7255ee5c5329fe19a5d96293a95788d9734a0a79829e8dc85ljjpk"
Oct 30 13:02:36.728761 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:02:36.728534    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ba8803d2-ea4e-4183-ab00-62f7b1fd85df-util\") pod \"61b1ff68c7255ee5c5329fe19a5d96293a95788d9734a0a79829e8dc85ljjpk\" (UID: \"ba8803d2-ea4e-4183-ab00-62f7b1fd85df\") " pod="openshift-marketplace/61b1ff68c7255ee5c5329fe19a5d96293a95788d9734a0a79829e8dc85ljjpk"
Oct 30 13:02:36.829409 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:02:36.829371    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tv8w4\" (UniqueName: \"kubernetes.io/projected/ba8803d2-ea4e-4183-ab00-62f7b1fd85df-kube-api-access-tv8w4\") pod \"61b1ff68c7255ee5c5329fe19a5d96293a95788d9734a0a79829e8dc85ljjpk\" (UID: \"ba8803d2-ea4e-4183-ab00-62f7b1fd85df\") " pod="openshift-marketplace/61b1ff68c7255ee5c5329fe19a5d96293a95788d9734a0a79829e8dc85ljjpk"
Oct 30 13:02:36.829576 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:02:36.829439    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ba8803d2-ea4e-4183-ab00-62f7b1fd85df-util\") pod \"61b1ff68c7255ee5c5329fe19a5d96293a95788d9734a0a79829e8dc85ljjpk\" (UID: \"ba8803d2-ea4e-4183-ab00-62f7b1fd85df\") " pod="openshift-marketplace/61b1ff68c7255ee5c5329fe19a5d96293a95788d9734a0a79829e8dc85ljjpk"
Oct 30 13:02:36.829576 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:02:36.829490    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ba8803d2-ea4e-4183-ab00-62f7b1fd85df-bundle\") pod \"61b1ff68c7255ee5c5329fe19a5d96293a95788d9734a0a79829e8dc85ljjpk\" (UID: \"ba8803d2-ea4e-4183-ab00-62f7b1fd85df\") " pod="openshift-marketplace/61b1ff68c7255ee5c5329fe19a5d96293a95788d9734a0a79829e8dc85ljjpk"
Oct 30 13:02:36.829890 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:02:36.829868    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ba8803d2-ea4e-4183-ab00-62f7b1fd85df-util\") pod \"61b1ff68c7255ee5c5329fe19a5d96293a95788d9734a0a79829e8dc85ljjpk\" (UID: \"ba8803d2-ea4e-4183-ab00-62f7b1fd85df\") " pod="openshift-marketplace/61b1ff68c7255ee5c5329fe19a5d96293a95788d9734a0a79829e8dc85ljjpk"
Oct 30 13:02:36.829890 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:02:36.829882    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ba8803d2-ea4e-4183-ab00-62f7b1fd85df-bundle\") pod \"61b1ff68c7255ee5c5329fe19a5d96293a95788d9734a0a79829e8dc85ljjpk\" (UID: \"ba8803d2-ea4e-4183-ab00-62f7b1fd85df\") " pod="openshift-marketplace/61b1ff68c7255ee5c5329fe19a5d96293a95788d9734a0a79829e8dc85ljjpk"
Oct 30 13:02:36.855248 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:02:36.855217    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tv8w4\" (UniqueName: \"kubernetes.io/projected/ba8803d2-ea4e-4183-ab00-62f7b1fd85df-kube-api-access-tv8w4\") pod \"61b1ff68c7255ee5c5329fe19a5d96293a95788d9734a0a79829e8dc85ljjpk\" (UID: \"ba8803d2-ea4e-4183-ab00-62f7b1fd85df\") " pod="openshift-marketplace/61b1ff68c7255ee5c5329fe19a5d96293a95788d9734a0a79829e8dc85ljjpk"
Oct 30 13:02:36.977916 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:02:36.977884    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/61b1ff68c7255ee5c5329fe19a5d96293a95788d9734a0a79829e8dc85ljjpk"
Oct 30 13:02:37.300206 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:02:37.300180    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/61b1ff68c7255ee5c5329fe19a5d96293a95788d9734a0a79829e8dc85ljjpk"]
Oct 30 13:02:37.301730 ip-10-0-142-84 kubenswrapper[4943]: W1030 13:02:37.301698    4943 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podba8803d2_ea4e_4183_ab00_62f7b1fd85df.slice/crio-ae096a1801e425e34a1a694feeccf3f782f6de83c8e38de8f296363b9e039565 WatchSource:0}: Error finding container ae096a1801e425e34a1a694feeccf3f782f6de83c8e38de8f296363b9e039565: Status 404 returned error can't find the container with id ae096a1801e425e34a1a694feeccf3f782f6de83c8e38de8f296363b9e039565
Oct 30 13:02:37.675093 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:02:37.675018    4943 generic.go:356] "Generic (PLEG): container finished" podID="ba8803d2-ea4e-4183-ab00-62f7b1fd85df" containerID="38792a16d392878ab977526789c4755d2293e648d70cc6f52aa10227e8b63db2" exitCode=0
Oct 30 13:02:37.675229 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:02:37.675105    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/61b1ff68c7255ee5c5329fe19a5d96293a95788d9734a0a79829e8dc85ljjpk" event={"ID":"ba8803d2-ea4e-4183-ab00-62f7b1fd85df","Type":"ContainerDied","Data":"38792a16d392878ab977526789c4755d2293e648d70cc6f52aa10227e8b63db2"}
Oct 30 13:02:37.675229 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:02:37.675157    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/61b1ff68c7255ee5c5329fe19a5d96293a95788d9734a0a79829e8dc85ljjpk" event={"ID":"ba8803d2-ea4e-4183-ab00-62f7b1fd85df","Type":"ContainerStarted","Data":"ae096a1801e425e34a1a694feeccf3f782f6de83c8e38de8f296363b9e039565"}
Oct 30 13:02:38.681353 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:02:38.681318    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-4xmnh" event={"ID":"d38d159c-a25a-4543-bf52-ca709fafbd69","Type":"ContainerStarted","Data":"006339bf13a0ff1aeddfe0fd10ad4d923526ad9a1beee29981156662cd7f4e6c"}
Oct 30 13:02:38.740200 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:02:38.740154    4943 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-4xmnh" podStartSLOduration=1.8139815320000001 podStartE2EDuration="4.740139284s" totalImagesPullingTime="2.926157752s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:02:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:02:38.738846396 +0000 UTC m=+334.526850129" watchObservedRunningTime="2026-10-30 13:02:38.740139284 +0000 UTC m=+334.528143013"
Oct 30 13:02:40.691119 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:02:40.691086    4943 generic.go:356] "Generic (PLEG): container finished" podID="ba8803d2-ea4e-4183-ab00-62f7b1fd85df" containerID="457b2c8121c2d73f6a85ae6538ede16904709e9089198fa77e42542fd68e1950" exitCode=0
Oct 30 13:02:40.691680 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:02:40.691144    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/61b1ff68c7255ee5c5329fe19a5d96293a95788d9734a0a79829e8dc85ljjpk" event={"ID":"ba8803d2-ea4e-4183-ab00-62f7b1fd85df","Type":"ContainerDied","Data":"457b2c8121c2d73f6a85ae6538ede16904709e9089198fa77e42542fd68e1950"}
Oct 30 13:02:41.697306 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:02:41.697247    4943 generic.go:356] "Generic (PLEG): container finished" podID="ba8803d2-ea4e-4183-ab00-62f7b1fd85df" containerID="756d4f4822d5ff199c2efef323cf1ad3ebbbd89387e5797ac98fb4cd57f1c4b0" exitCode=0
Oct 30 13:02:41.697306 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:02:41.697305    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/61b1ff68c7255ee5c5329fe19a5d96293a95788d9734a0a79829e8dc85ljjpk" event={"ID":"ba8803d2-ea4e-4183-ab00-62f7b1fd85df","Type":"ContainerDied","Data":"756d4f4822d5ff199c2efef323cf1ad3ebbbd89387e5797ac98fb4cd57f1c4b0"}
Oct 30 13:02:42.820783 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:02:42.820759    4943 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/61b1ff68c7255ee5c5329fe19a5d96293a95788d9734a0a79829e8dc85ljjpk"
Oct 30 13:02:42.880092 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:02:42.879714    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ba8803d2-ea4e-4183-ab00-62f7b1fd85df-bundle\" (UniqueName: \"kubernetes.io/empty-dir/ba8803d2-ea4e-4183-ab00-62f7b1fd85df-bundle\") pod \"ba8803d2-ea4e-4183-ab00-62f7b1fd85df\" (UID: \"ba8803d2-ea4e-4183-ab00-62f7b1fd85df\") "
Oct 30 13:02:42.880092 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:02:42.879956    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ba8803d2-ea4e-4183-ab00-62f7b1fd85df-util\" (UniqueName: \"kubernetes.io/empty-dir/ba8803d2-ea4e-4183-ab00-62f7b1fd85df-util\") pod \"ba8803d2-ea4e-4183-ab00-62f7b1fd85df\" (UID: \"ba8803d2-ea4e-4183-ab00-62f7b1fd85df\") "
Oct 30 13:02:42.880328 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:02:42.880298    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ba8803d2-ea4e-4183-ab00-62f7b1fd85df-bundle" pod "ba8803d2-ea4e-4183-ab00-62f7b1fd85df" (UID: "ba8803d2-ea4e-4183-ab00-62f7b1fd85df"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:02:42.880462 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:02:42.880436    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ba8803d2-ea4e-4183-ab00-62f7b1fd85df-kube-api-access-tv8w4\" (UniqueName: \"kubernetes.io/projected/ba8803d2-ea4e-4183-ab00-62f7b1fd85df-kube-api-access-tv8w4\") pod \"ba8803d2-ea4e-4183-ab00-62f7b1fd85df\" (UID: \"ba8803d2-ea4e-4183-ab00-62f7b1fd85df\") "
Oct 30 13:02:42.880964 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:02:42.880945    4943 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ba8803d2-ea4e-4183-ab00-62f7b1fd85df-bundle\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:02:42.885672 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:02:42.883559    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ba8803d2-ea4e-4183-ab00-62f7b1fd85df-kube-api-access-tv8w4" pod "ba8803d2-ea4e-4183-ab00-62f7b1fd85df" (UID: "ba8803d2-ea4e-4183-ab00-62f7b1fd85df"). InnerVolumeSpecName "kube-api-access-tv8w4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:02:42.910872 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:02:42.910846    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ba8803d2-ea4e-4183-ab00-62f7b1fd85df-util" pod "ba8803d2-ea4e-4183-ab00-62f7b1fd85df" (UID: "ba8803d2-ea4e-4183-ab00-62f7b1fd85df"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:02:42.981433 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:02:42.981386    4943 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ba8803d2-ea4e-4183-ab00-62f7b1fd85df-util\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:02:42.981433 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:02:42.981405    4943 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tv8w4\" (UniqueName: \"kubernetes.io/projected/ba8803d2-ea4e-4183-ab00-62f7b1fd85df-kube-api-access-tv8w4\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:02:43.706494 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:02:43.706454    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/61b1ff68c7255ee5c5329fe19a5d96293a95788d9734a0a79829e8dc85ljjpk" event={"ID":"ba8803d2-ea4e-4183-ab00-62f7b1fd85df","Type":"ContainerDied","Data":"ae096a1801e425e34a1a694feeccf3f782f6de83c8e38de8f296363b9e039565"}
Oct 30 13:02:43.706661 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:02:43.706513    4943 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ae096a1801e425e34a1a694feeccf3f782f6de83c8e38de8f296363b9e039565"
Oct 30 13:02:43.706661 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:02:43.706466    4943 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/61b1ff68c7255ee5c5329fe19a5d96293a95788d9734a0a79829e8dc85ljjpk"
Oct 30 13:03:05.953913 ip-10-0-142-84 kubenswrapper[4943]: E1030 13:03:05.953874    4943 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-knqz6" containerName="extractor"
Oct 30 13:03:13.399711 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:03:13.399676    4943 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71rdpvf"]
Oct 30 13:03:13.400330 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:03:13.400092    4943 cpu_manager.go:414] "containerMap: removing container" podUID="ba8803d2-ea4e-4183-ab00-62f7b1fd85df" containerName="util"
Oct 30 13:03:13.400330 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:03:13.400110    4943 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ba8803d2-ea4e-4183-ab00-62f7b1fd85df" containerName="util"
Oct 30 13:03:13.400330 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:03:13.400136    4943 cpu_manager.go:414] "containerMap: removing container" podUID="ba8803d2-ea4e-4183-ab00-62f7b1fd85df" containerName="extract"
Oct 30 13:03:13.400330 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:03:13.400144    4943 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ba8803d2-ea4e-4183-ab00-62f7b1fd85df" containerName="extract"
Oct 30 13:03:13.400330 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:03:13.400160    4943 cpu_manager.go:414] "containerMap: removing container" podUID="ba8803d2-ea4e-4183-ab00-62f7b1fd85df" containerName="pull"
Oct 30 13:03:13.400330 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:03:13.400171    4943 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ba8803d2-ea4e-4183-ab00-62f7b1fd85df" containerName="pull"
Oct 30 13:03:13.400330 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:03:13.400287    4943 memory_manager.go:365] "RemoveStaleState removing state" podUID="ba8803d2-ea4e-4183-ab00-62f7b1fd85df" containerName="extract"
Oct 30 13:03:13.404909 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:03:13.404887    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71rdpvf"
Oct 30 13:03:13.428834 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:03:13.428810    4943 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Oct 30 13:03:13.428960 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:03:13.428855    4943 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Oct 30 13:03:13.443005 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:03:13.442981    4943 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"e644421527c169d876fdb59235c927986d6bd2-9d2815fc-dockercfg-w6zgf\""
Oct 30 13:03:13.444328 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:03:13.444309    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71rdpvf"]
Oct 30 13:03:13.493722 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:03:13.493698    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0cb6abcc-0914-40ee-992b-f94f5f7729c7-bundle\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71rdpvf\" (UID: \"0cb6abcc-0914-40ee-992b-f94f5f7729c7\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71rdpvf"
Oct 30 13:03:13.493829 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:03:13.493741    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0cb6abcc-0914-40ee-992b-f94f5f7729c7-util\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71rdpvf\" (UID: \"0cb6abcc-0914-40ee-992b-f94f5f7729c7\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71rdpvf"
Oct 30 13:03:13.493829 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:03:13.493760    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-thpfw\" (UniqueName: \"kubernetes.io/projected/0cb6abcc-0914-40ee-992b-f94f5f7729c7-kube-api-access-thpfw\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71rdpvf\" (UID: \"0cb6abcc-0914-40ee-992b-f94f5f7729c7\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71rdpvf"
Oct 30 13:03:13.594816 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:03:13.594775    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0cb6abcc-0914-40ee-992b-f94f5f7729c7-bundle\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71rdpvf\" (UID: \"0cb6abcc-0914-40ee-992b-f94f5f7729c7\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71rdpvf"
Oct 30 13:03:13.594970 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:03:13.594843    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0cb6abcc-0914-40ee-992b-f94f5f7729c7-util\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71rdpvf\" (UID: \"0cb6abcc-0914-40ee-992b-f94f5f7729c7\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71rdpvf"
Oct 30 13:03:13.594970 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:03:13.594864    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-thpfw\" (UniqueName: \"kubernetes.io/projected/0cb6abcc-0914-40ee-992b-f94f5f7729c7-kube-api-access-thpfw\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71rdpvf\" (UID: \"0cb6abcc-0914-40ee-992b-f94f5f7729c7\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71rdpvf"
Oct 30 13:03:13.595187 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:03:13.595169    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0cb6abcc-0914-40ee-992b-f94f5f7729c7-bundle\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71rdpvf\" (UID: \"0cb6abcc-0914-40ee-992b-f94f5f7729c7\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71rdpvf"
Oct 30 13:03:13.595227 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:03:13.595197    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0cb6abcc-0914-40ee-992b-f94f5f7729c7-util\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71rdpvf\" (UID: \"0cb6abcc-0914-40ee-992b-f94f5f7729c7\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71rdpvf"
Oct 30 13:03:13.631339 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:03:13.631309    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-thpfw\" (UniqueName: \"kubernetes.io/projected/0cb6abcc-0914-40ee-992b-f94f5f7729c7-kube-api-access-thpfw\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71rdpvf\" (UID: \"0cb6abcc-0914-40ee-992b-f94f5f7729c7\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71rdpvf"
Oct 30 13:03:13.717549 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:03:13.717527    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71rdpvf"
Oct 30 13:03:14.025530 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:03:14.025409    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71rdpvf"]
Oct 30 13:03:14.028246 ip-10-0-142-84 kubenswrapper[4943]: W1030 13:03:14.028220    4943 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0cb6abcc_0914_40ee_992b_f94f5f7729c7.slice/crio-3941a225a168f4809a9afde76f9ea62909ffec87b588957b147f954c38d74840 WatchSource:0}: Error finding container 3941a225a168f4809a9afde76f9ea62909ffec87b588957b147f954c38d74840: Status 404 returned error can't find the container with id 3941a225a168f4809a9afde76f9ea62909ffec87b588957b147f954c38d74840
Oct 30 13:03:14.833648 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:03:14.833614    4943 generic.go:356] "Generic (PLEG): container finished" podID="0cb6abcc-0914-40ee-992b-f94f5f7729c7" containerID="2d46adc519110bac06bf80f275c33333d4748a09c7e17bdb5de9e9b93899ef9b" exitCode=0
Oct 30 13:03:14.834130 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:03:14.833672    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71rdpvf" event={"ID":"0cb6abcc-0914-40ee-992b-f94f5f7729c7","Type":"ContainerDied","Data":"2d46adc519110bac06bf80f275c33333d4748a09c7e17bdb5de9e9b93899ef9b"}
Oct 30 13:03:14.834130 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:03:14.833752    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71rdpvf" event={"ID":"0cb6abcc-0914-40ee-992b-f94f5f7729c7","Type":"ContainerStarted","Data":"3941a225a168f4809a9afde76f9ea62909ffec87b588957b147f954c38d74840"}
Oct 30 13:03:15.839534 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:03:15.839502    4943 generic.go:356] "Generic (PLEG): container finished" podID="0cb6abcc-0914-40ee-992b-f94f5f7729c7" containerID="e1fd8f437e71426c5381ad0e1ab6c26d1ea6074191460941c1434acc0b9b4da6" exitCode=0
Oct 30 13:03:15.839898 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:03:15.839563    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71rdpvf" event={"ID":"0cb6abcc-0914-40ee-992b-f94f5f7729c7","Type":"ContainerDied","Data":"e1fd8f437e71426c5381ad0e1ab6c26d1ea6074191460941c1434acc0b9b4da6"}
Oct 30 13:03:16.845773 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:03:16.845740    4943 generic.go:356] "Generic (PLEG): container finished" podID="0cb6abcc-0914-40ee-992b-f94f5f7729c7" containerID="c9849cec015b8f2647e2e75963e6d4750952c43296cb1246ebc1643def6c21bc" exitCode=0
Oct 30 13:03:16.846156 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:03:16.845804    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71rdpvf" event={"ID":"0cb6abcc-0914-40ee-992b-f94f5f7729c7","Type":"ContainerDied","Data":"c9849cec015b8f2647e2e75963e6d4750952c43296cb1246ebc1643def6c21bc"}
Oct 30 13:03:17.978300 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:03:17.978253    4943 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71rdpvf"
Oct 30 13:03:18.028039 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:03:18.028010    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0cb6abcc-0914-40ee-992b-f94f5f7729c7-kube-api-access-thpfw\" (UniqueName: \"kubernetes.io/projected/0cb6abcc-0914-40ee-992b-f94f5f7729c7-kube-api-access-thpfw\") pod \"0cb6abcc-0914-40ee-992b-f94f5f7729c7\" (UID: \"0cb6abcc-0914-40ee-992b-f94f5f7729c7\") "
Oct 30 13:03:18.028169 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:03:18.028074    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0cb6abcc-0914-40ee-992b-f94f5f7729c7-util\" (UniqueName: \"kubernetes.io/empty-dir/0cb6abcc-0914-40ee-992b-f94f5f7729c7-util\") pod \"0cb6abcc-0914-40ee-992b-f94f5f7729c7\" (UID: \"0cb6abcc-0914-40ee-992b-f94f5f7729c7\") "
Oct 30 13:03:18.028169 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:03:18.028149    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0cb6abcc-0914-40ee-992b-f94f5f7729c7-bundle\" (UniqueName: \"kubernetes.io/empty-dir/0cb6abcc-0914-40ee-992b-f94f5f7729c7-bundle\") pod \"0cb6abcc-0914-40ee-992b-f94f5f7729c7\" (UID: \"0cb6abcc-0914-40ee-992b-f94f5f7729c7\") "
Oct 30 13:03:18.028897 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:03:18.028865    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0cb6abcc-0914-40ee-992b-f94f5f7729c7-bundle" pod "0cb6abcc-0914-40ee-992b-f94f5f7729c7" (UID: "0cb6abcc-0914-40ee-992b-f94f5f7729c7"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:03:18.030503 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:03:18.030455    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0cb6abcc-0914-40ee-992b-f94f5f7729c7-kube-api-access-thpfw" pod "0cb6abcc-0914-40ee-992b-f94f5f7729c7" (UID: "0cb6abcc-0914-40ee-992b-f94f5f7729c7"). InnerVolumeSpecName "kube-api-access-thpfw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:03:18.034155 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:03:18.034127    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0cb6abcc-0914-40ee-992b-f94f5f7729c7-util" pod "0cb6abcc-0914-40ee-992b-f94f5f7729c7" (UID: "0cb6abcc-0914-40ee-992b-f94f5f7729c7"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:03:18.129564 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:03:18.129505    4943 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0cb6abcc-0914-40ee-992b-f94f5f7729c7-util\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:03:18.129564 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:03:18.129529    4943 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0cb6abcc-0914-40ee-992b-f94f5f7729c7-bundle\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:03:18.129564 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:03:18.129541    4943 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-thpfw\" (UniqueName: \"kubernetes.io/projected/0cb6abcc-0914-40ee-992b-f94f5f7729c7-kube-api-access-thpfw\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:03:18.856052 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:03:18.856008    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71rdpvf" event={"ID":"0cb6abcc-0914-40ee-992b-f94f5f7729c7","Type":"ContainerDied","Data":"3941a225a168f4809a9afde76f9ea62909ffec87b588957b147f954c38d74840"}
Oct 30 13:03:18.856231 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:03:18.856070    4943 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3941a225a168f4809a9afde76f9ea62909ffec87b588957b147f954c38d74840"
Oct 30 13:03:18.856231 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:03:18.856033    4943 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71rdpvf"
Oct 30 13:03:28.414321 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:03:28.414284    4943 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-64d4d4766b-5rmd7"]
Oct 30 13:03:28.414688 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:03:28.414571    4943 cpu_manager.go:414] "containerMap: removing container" podUID="0cb6abcc-0914-40ee-992b-f94f5f7729c7" containerName="util"
Oct 30 13:03:28.414688 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:03:28.414583    4943 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0cb6abcc-0914-40ee-992b-f94f5f7729c7" containerName="util"
Oct 30 13:03:28.414688 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:03:28.414607    4943 cpu_manager.go:414] "containerMap: removing container" podUID="0cb6abcc-0914-40ee-992b-f94f5f7729c7" containerName="pull"
Oct 30 13:03:28.414688 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:03:28.414612    4943 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0cb6abcc-0914-40ee-992b-f94f5f7729c7" containerName="pull"
Oct 30 13:03:28.414688 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:03:28.414628    4943 cpu_manager.go:414] "containerMap: removing container" podUID="0cb6abcc-0914-40ee-992b-f94f5f7729c7" containerName="extract"
Oct 30 13:03:28.414688 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:03:28.414633    4943 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0cb6abcc-0914-40ee-992b-f94f5f7729c7" containerName="extract"
Oct 30 13:03:28.414688 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:03:28.414682    4943 memory_manager.go:365] "RemoveStaleState removing state" podUID="0cb6abcc-0914-40ee-992b-f94f5f7729c7" containerName="extract"
Oct 30 13:03:28.417741 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:03:28.417722    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-64d4d4766b-5rmd7"
Oct 30 13:03:28.459298 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:03:28.459247    4943 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Oct 30 13:03:28.459298 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:03:28.459252    4943 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Oct 30 13:03:28.459298 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:03:28.459279    4943 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Oct 30 13:03:28.459514 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:03:28.459305    4943 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Oct 30 13:03:28.459514 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:03:28.459366    4943 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-tp27k\""
Oct 30 13:03:28.459514 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:03:28.459424    4943 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-metrics-tls\""
Oct 30 13:03:28.463687 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:03:28.463663    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-64d4d4766b-5rmd7"]
Oct 30 13:03:28.510720 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:03:28.510691    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wv2k6\" (UniqueName: \"kubernetes.io/projected/0d20cc9a-79da-4271-9b8e-8d667fd813ba-kube-api-access-wv2k6\") pod \"opendatahub-operator-controller-manager-64d4d4766b-5rmd7\" (UID: \"0d20cc9a-79da-4271-9b8e-8d667fd813ba\") " pod="opendatahub/opendatahub-operator-controller-manager-64d4d4766b-5rmd7"
Oct 30 13:03:28.510843 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:03:28.510734    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0d20cc9a-79da-4271-9b8e-8d667fd813ba-tmp\") pod \"opendatahub-operator-controller-manager-64d4d4766b-5rmd7\" (UID: \"0d20cc9a-79da-4271-9b8e-8d667fd813ba\") " pod="opendatahub/opendatahub-operator-controller-manager-64d4d4766b-5rmd7"
Oct 30 13:03:28.510843 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:03:28.510763    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/0d20cc9a-79da-4271-9b8e-8d667fd813ba-metrics-cert\") pod \"opendatahub-operator-controller-manager-64d4d4766b-5rmd7\" (UID: \"0d20cc9a-79da-4271-9b8e-8d667fd813ba\") " pod="opendatahub/opendatahub-operator-controller-manager-64d4d4766b-5rmd7"
Oct 30 13:03:28.510843 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:03:28.510824    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/0d20cc9a-79da-4271-9b8e-8d667fd813ba-apiservice-cert\") pod \"opendatahub-operator-controller-manager-64d4d4766b-5rmd7\" (UID: \"0d20cc9a-79da-4271-9b8e-8d667fd813ba\") " pod="opendatahub/opendatahub-operator-controller-manager-64d4d4766b-5rmd7"
Oct 30 13:03:28.510956 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:03:28.510858    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0d20cc9a-79da-4271-9b8e-8d667fd813ba-webhook-cert\") pod \"opendatahub-operator-controller-manager-64d4d4766b-5rmd7\" (UID: \"0d20cc9a-79da-4271-9b8e-8d667fd813ba\") " pod="opendatahub/opendatahub-operator-controller-manager-64d4d4766b-5rmd7"
Oct 30 13:03:28.510956 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:03:28.510902    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/0d20cc9a-79da-4271-9b8e-8d667fd813ba-manifests\") pod \"opendatahub-operator-controller-manager-64d4d4766b-5rmd7\" (UID: \"0d20cc9a-79da-4271-9b8e-8d667fd813ba\") " pod="opendatahub/opendatahub-operator-controller-manager-64d4d4766b-5rmd7"
Oct 30 13:03:28.611442 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:03:28.611404    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/0d20cc9a-79da-4271-9b8e-8d667fd813ba-metrics-cert\") pod \"opendatahub-operator-controller-manager-64d4d4766b-5rmd7\" (UID: \"0d20cc9a-79da-4271-9b8e-8d667fd813ba\") " pod="opendatahub/opendatahub-operator-controller-manager-64d4d4766b-5rmd7"
Oct 30 13:03:28.611442 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:03:28.611443    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/0d20cc9a-79da-4271-9b8e-8d667fd813ba-apiservice-cert\") pod \"opendatahub-operator-controller-manager-64d4d4766b-5rmd7\" (UID: \"0d20cc9a-79da-4271-9b8e-8d667fd813ba\") " pod="opendatahub/opendatahub-operator-controller-manager-64d4d4766b-5rmd7"
Oct 30 13:03:28.611707 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:03:28.611461    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0d20cc9a-79da-4271-9b8e-8d667fd813ba-webhook-cert\") pod \"opendatahub-operator-controller-manager-64d4d4766b-5rmd7\" (UID: \"0d20cc9a-79da-4271-9b8e-8d667fd813ba\") " pod="opendatahub/opendatahub-operator-controller-manager-64d4d4766b-5rmd7"
Oct 30 13:03:28.611707 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:03:28.611496    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/0d20cc9a-79da-4271-9b8e-8d667fd813ba-manifests\") pod \"opendatahub-operator-controller-manager-64d4d4766b-5rmd7\" (UID: \"0d20cc9a-79da-4271-9b8e-8d667fd813ba\") " pod="opendatahub/opendatahub-operator-controller-manager-64d4d4766b-5rmd7"
Oct 30 13:03:28.611707 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:03:28.611524    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wv2k6\" (UniqueName: \"kubernetes.io/projected/0d20cc9a-79da-4271-9b8e-8d667fd813ba-kube-api-access-wv2k6\") pod \"opendatahub-operator-controller-manager-64d4d4766b-5rmd7\" (UID: \"0d20cc9a-79da-4271-9b8e-8d667fd813ba\") " pod="opendatahub/opendatahub-operator-controller-manager-64d4d4766b-5rmd7"
Oct 30 13:03:28.611707 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:03:28.611556    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0d20cc9a-79da-4271-9b8e-8d667fd813ba-tmp\") pod \"opendatahub-operator-controller-manager-64d4d4766b-5rmd7\" (UID: \"0d20cc9a-79da-4271-9b8e-8d667fd813ba\") " pod="opendatahub/opendatahub-operator-controller-manager-64d4d4766b-5rmd7"
Oct 30 13:03:28.611990 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:03:28.611962    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/0d20cc9a-79da-4271-9b8e-8d667fd813ba-manifests\") pod \"opendatahub-operator-controller-manager-64d4d4766b-5rmd7\" (UID: \"0d20cc9a-79da-4271-9b8e-8d667fd813ba\") " pod="opendatahub/opendatahub-operator-controller-manager-64d4d4766b-5rmd7"
Oct 30 13:03:28.614025 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:03:28.613992    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0d20cc9a-79da-4271-9b8e-8d667fd813ba-webhook-cert\") pod \"opendatahub-operator-controller-manager-64d4d4766b-5rmd7\" (UID: \"0d20cc9a-79da-4271-9b8e-8d667fd813ba\") " pod="opendatahub/opendatahub-operator-controller-manager-64d4d4766b-5rmd7"
Oct 30 13:03:28.614158 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:03:28.614057    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/0d20cc9a-79da-4271-9b8e-8d667fd813ba-metrics-cert\") pod \"opendatahub-operator-controller-manager-64d4d4766b-5rmd7\" (UID: \"0d20cc9a-79da-4271-9b8e-8d667fd813ba\") " pod="opendatahub/opendatahub-operator-controller-manager-64d4d4766b-5rmd7"
Oct 30 13:03:28.614229 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:03:28.614163    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0d20cc9a-79da-4271-9b8e-8d667fd813ba-tmp\") pod \"opendatahub-operator-controller-manager-64d4d4766b-5rmd7\" (UID: \"0d20cc9a-79da-4271-9b8e-8d667fd813ba\") " pod="opendatahub/opendatahub-operator-controller-manager-64d4d4766b-5rmd7"
Oct 30 13:03:28.614466 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:03:28.614446    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/0d20cc9a-79da-4271-9b8e-8d667fd813ba-apiservice-cert\") pod \"opendatahub-operator-controller-manager-64d4d4766b-5rmd7\" (UID: \"0d20cc9a-79da-4271-9b8e-8d667fd813ba\") " pod="opendatahub/opendatahub-operator-controller-manager-64d4d4766b-5rmd7"
Oct 30 13:03:28.641774 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:03:28.641744    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wv2k6\" (UniqueName: \"kubernetes.io/projected/0d20cc9a-79da-4271-9b8e-8d667fd813ba-kube-api-access-wv2k6\") pod \"opendatahub-operator-controller-manager-64d4d4766b-5rmd7\" (UID: \"0d20cc9a-79da-4271-9b8e-8d667fd813ba\") " pod="opendatahub/opendatahub-operator-controller-manager-64d4d4766b-5rmd7"
Oct 30 13:03:28.731792 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:03:28.731766    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-64d4d4766b-5rmd7"
Oct 30 13:03:29.247653 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:03:29.247612    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-64d4d4766b-5rmd7"]
Oct 30 13:03:29.263642 ip-10-0-142-84 kubenswrapper[4943]: W1030 13:03:29.263614    4943 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0d20cc9a_79da_4271_9b8e_8d667fd813ba.slice/crio-911daf131b4ae901a5e9339fd6094b6d858241a95e5e56b4bc6ca98170667f57 WatchSource:0}: Error finding container 911daf131b4ae901a5e9339fd6094b6d858241a95e5e56b4bc6ca98170667f57: Status 404 returned error can't find the container with id 911daf131b4ae901a5e9339fd6094b6d858241a95e5e56b4bc6ca98170667f57
Oct 30 13:03:29.903475 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:03:29.903435    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-64d4d4766b-5rmd7" event={"ID":"0d20cc9a-79da-4271-9b8e-8d667fd813ba","Type":"ContainerStarted","Data":"911daf131b4ae901a5e9339fd6094b6d858241a95e5e56b4bc6ca98170667f57"}
Oct 30 13:03:32.919709 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:03:32.919624    4943 generic.go:356] "Generic (PLEG): container finished" podID="0d20cc9a-79da-4271-9b8e-8d667fd813ba" containerID="39cde33c3a898226962853f3821a71d2a8d861747c0f40920df518fdd621fbd9" exitCode=0
Oct 30 13:03:32.920060 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:03:32.919712    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-64d4d4766b-5rmd7" event={"ID":"0d20cc9a-79da-4271-9b8e-8d667fd813ba","Type":"ContainerDied","Data":"39cde33c3a898226962853f3821a71d2a8d861747c0f40920df518fdd621fbd9"}
Oct 30 13:03:33.925706 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:03:33.925669    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-64d4d4766b-5rmd7" event={"ID":"0d20cc9a-79da-4271-9b8e-8d667fd813ba","Type":"ContainerStarted","Data":"3a25a98c848147988c8e21ad6e3541e8024704a52429541073da3839d89baedd"}
Oct 30 13:03:33.926110 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:03:33.925784    4943 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-64d4d4766b-5rmd7"
Oct 30 13:03:34.139157 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:03:34.139107    4943 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-64d4d4766b-5rmd7" podStartSLOduration=2.531629417 podStartE2EDuration="6.139092003s" totalImagesPullingTime="3.168569047s" totalInitContainerRuntime="438.893539ms" isStatefulPod=false podCreationTimestamp="2026-10-30 13:03:28 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:03:34.1244815 +0000 UTC m=+389.912485229" watchObservedRunningTime="2026-10-30 13:03:34.139092003 +0000 UTC m=+389.927095733"
Oct 30 13:03:35.953798 ip-10-0-142-84 kubenswrapper[4943]: E1030 13:03:35.953757    4943 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-knqz6" containerName="extractor"
Oct 30 13:03:44.932159 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:03:44.932126    4943 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-64d4d4766b-5rmd7"
Oct 30 13:03:51.998352 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:03:51.998317    4943 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/odh-observability-584f99c788-mwqh2"]
Oct 30 13:03:52.003384 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:03:52.003364    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-observability-584f99c788-mwqh2"
Oct 30 13:03:52.024015 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:03:52.023983    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-observability-584f99c788-mwqh2"]
Oct 30 13:03:52.036170 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:03:52.036146    4943 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-observability-webhook-cert\""
Oct 30 13:03:52.036170 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:03:52.036149    4943 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-observability-dockercfg-77htm\""
Oct 30 13:03:52.095692 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:03:52.095663    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-certs\" (UniqueName: \"kubernetes.io/secret/310f7577-cbbe-4321-98eb-6d65bdc9cad2-webhook-certs\") pod \"odh-observability-584f99c788-mwqh2\" (UID: \"310f7577-cbbe-4321-98eb-6d65bdc9cad2\") " pod="opendatahub/odh-observability-584f99c788-mwqh2"
Oct 30 13:03:52.095821 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:03:52.095706    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-brn8s\" (UniqueName: \"kubernetes.io/projected/310f7577-cbbe-4321-98eb-6d65bdc9cad2-kube-api-access-brn8s\") pod \"odh-observability-584f99c788-mwqh2\" (UID: \"310f7577-cbbe-4321-98eb-6d65bdc9cad2\") " pod="opendatahub/odh-observability-584f99c788-mwqh2"
Oct 30 13:03:52.196421 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:03:52.196394    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-certs\" (UniqueName: \"kubernetes.io/secret/310f7577-cbbe-4321-98eb-6d65bdc9cad2-webhook-certs\") pod \"odh-observability-584f99c788-mwqh2\" (UID: \"310f7577-cbbe-4321-98eb-6d65bdc9cad2\") " pod="opendatahub/odh-observability-584f99c788-mwqh2"
Oct 30 13:03:52.196564 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:03:52.196440    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-brn8s\" (UniqueName: \"kubernetes.io/projected/310f7577-cbbe-4321-98eb-6d65bdc9cad2-kube-api-access-brn8s\") pod \"odh-observability-584f99c788-mwqh2\" (UID: \"310f7577-cbbe-4321-98eb-6d65bdc9cad2\") " pod="opendatahub/odh-observability-584f99c788-mwqh2"
Oct 30 13:03:52.196564 ip-10-0-142-84 kubenswrapper[4943]: E1030 13:03:52.196527    4943 secret.go:189] Couldn't get secret opendatahub/odh-observability-webhook-cert: secret "odh-observability-webhook-cert" not found
Oct 30 13:03:52.196678 ip-10-0-142-84 kubenswrapper[4943]: E1030 13:03:52.196602    4943 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/310f7577-cbbe-4321-98eb-6d65bdc9cad2-webhook-certs podName:310f7577-cbbe-4321-98eb-6d65bdc9cad2 nodeName:}" failed. No retries permitted until 2026-10-30 13:03:52.69658078 +0000 UTC m=+408.484584505 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "webhook-certs" (UniqueName: "kubernetes.io/secret/310f7577-cbbe-4321-98eb-6d65bdc9cad2-webhook-certs") pod "odh-observability-584f99c788-mwqh2" (UID: "310f7577-cbbe-4321-98eb-6d65bdc9cad2") : secret "odh-observability-webhook-cert" not found
Oct 30 13:03:52.224376 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:03:52.224346    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-brn8s\" (UniqueName: \"kubernetes.io/projected/310f7577-cbbe-4321-98eb-6d65bdc9cad2-kube-api-access-brn8s\") pod \"odh-observability-584f99c788-mwqh2\" (UID: \"310f7577-cbbe-4321-98eb-6d65bdc9cad2\") " pod="opendatahub/odh-observability-584f99c788-mwqh2"
Oct 30 13:03:52.700695 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:03:52.700662    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-certs\" (UniqueName: \"kubernetes.io/secret/310f7577-cbbe-4321-98eb-6d65bdc9cad2-webhook-certs\") pod \"odh-observability-584f99c788-mwqh2\" (UID: \"310f7577-cbbe-4321-98eb-6d65bdc9cad2\") " pod="opendatahub/odh-observability-584f99c788-mwqh2"
Oct 30 13:03:52.703029 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:03:52.703008    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-certs\" (UniqueName: \"kubernetes.io/secret/310f7577-cbbe-4321-98eb-6d65bdc9cad2-webhook-certs\") pod \"odh-observability-584f99c788-mwqh2\" (UID: \"310f7577-cbbe-4321-98eb-6d65bdc9cad2\") " pod="opendatahub/odh-observability-584f99c788-mwqh2"
Oct 30 13:03:52.915042 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:03:52.915009    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-observability-584f99c788-mwqh2"
Oct 30 13:03:53.212607 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:03:53.212581    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-observability-584f99c788-mwqh2"]
Oct 30 13:03:53.214651 ip-10-0-142-84 kubenswrapper[4943]: W1030 13:03:53.214620    4943 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod310f7577_cbbe_4321_98eb_6d65bdc9cad2.slice/crio-92717af461bf84e167f3b615ab23dae0b6cae444e7517bf1ed8bdf549936311c WatchSource:0}: Error finding container 92717af461bf84e167f3b615ab23dae0b6cae444e7517bf1ed8bdf549936311c: Status 404 returned error can't find the container with id 92717af461bf84e167f3b615ab23dae0b6cae444e7517bf1ed8bdf549936311c
Oct 30 13:03:54.009901 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:03:54.009824    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-observability-584f99c788-mwqh2" event={"ID":"310f7577-cbbe-4321-98eb-6d65bdc9cad2","Type":"ContainerStarted","Data":"92717af461bf84e167f3b615ab23dae0b6cae444e7517bf1ed8bdf549936311c"}
Oct 30 13:03:56.019947 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:03:56.019908    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-observability-584f99c788-mwqh2" event={"ID":"310f7577-cbbe-4321-98eb-6d65bdc9cad2","Type":"ContainerStarted","Data":"121f6ce5b5d12ea2e6d6cda4e68f645e904aa2fba5e63fffa627e77bcec5889f"}
Oct 30 13:03:56.020442 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:03:56.020040    4943 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-observability-584f99c788-mwqh2"
Oct 30 13:03:56.067248 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:03:56.067187    4943 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/odh-observability-584f99c788-mwqh2" podStartSLOduration=2.431528521 podStartE2EDuration="5.067174946s" totalImagesPullingTime="2.635646425s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:03:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:03:56.065812292 +0000 UTC m=+411.853816024" watchObservedRunningTime="2026-10-30 13:03:56.067174946 +0000 UTC m=+411.855178675"
Oct 30 13:03:59.745990 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:03:59.745957    4943 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-75fdd59bb7-cpmzt"]
Oct 30 13:03:59.749364 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:03:59.749342    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-75fdd59bb7-cpmzt"
Oct 30 13:03:59.762657 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:03:59.762632    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e4cc43ab-deda-4426-83f5-236f95bfdfc4-tls-certs\") pod \"kube-auth-proxy-75fdd59bb7-cpmzt\" (UID: \"e4cc43ab-deda-4426-83f5-236f95bfdfc4\") " pod="openshift-ingress/kube-auth-proxy-75fdd59bb7-cpmzt"
Oct 30 13:03:59.762770 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:03:59.762666    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e4cc43ab-deda-4426-83f5-236f95bfdfc4-tmp\") pod \"kube-auth-proxy-75fdd59bb7-cpmzt\" (UID: \"e4cc43ab-deda-4426-83f5-236f95bfdfc4\") " pod="openshift-ingress/kube-auth-proxy-75fdd59bb7-cpmzt"
Oct 30 13:03:59.762770 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:03:59.762689    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vcccx\" (UniqueName: \"kubernetes.io/projected/e4cc43ab-deda-4426-83f5-236f95bfdfc4-kube-api-access-vcccx\") pod \"kube-auth-proxy-75fdd59bb7-cpmzt\" (UID: \"e4cc43ab-deda-4426-83f5-236f95bfdfc4\") " pod="openshift-ingress/kube-auth-proxy-75fdd59bb7-cpmzt"
Oct 30 13:03:59.771139 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:03:59.771118    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-75fdd59bb7-cpmzt"]
Oct 30 13:03:59.783467 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:03:59.783445    4943 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Oct 30 13:03:59.783467 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:03:59.783448    4943 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Oct 30 13:03:59.783624 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:03:59.783529    4943 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-wlkpx\""
Oct 30 13:03:59.863899 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:03:59.863874    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e4cc43ab-deda-4426-83f5-236f95bfdfc4-tls-certs\") pod \"kube-auth-proxy-75fdd59bb7-cpmzt\" (UID: \"e4cc43ab-deda-4426-83f5-236f95bfdfc4\") " pod="openshift-ingress/kube-auth-proxy-75fdd59bb7-cpmzt"
Oct 30 13:03:59.864037 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:03:59.863908    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e4cc43ab-deda-4426-83f5-236f95bfdfc4-tmp\") pod \"kube-auth-proxy-75fdd59bb7-cpmzt\" (UID: \"e4cc43ab-deda-4426-83f5-236f95bfdfc4\") " pod="openshift-ingress/kube-auth-proxy-75fdd59bb7-cpmzt"
Oct 30 13:03:59.864037 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:03:59.863933    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vcccx\" (UniqueName: \"kubernetes.io/projected/e4cc43ab-deda-4426-83f5-236f95bfdfc4-kube-api-access-vcccx\") pod \"kube-auth-proxy-75fdd59bb7-cpmzt\" (UID: \"e4cc43ab-deda-4426-83f5-236f95bfdfc4\") " pod="openshift-ingress/kube-auth-proxy-75fdd59bb7-cpmzt"
Oct 30 13:03:59.866056 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:03:59.866035    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e4cc43ab-deda-4426-83f5-236f95bfdfc4-tmp\") pod \"kube-auth-proxy-75fdd59bb7-cpmzt\" (UID: \"e4cc43ab-deda-4426-83f5-236f95bfdfc4\") " pod="openshift-ingress/kube-auth-proxy-75fdd59bb7-cpmzt"
Oct 30 13:03:59.866419 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:03:59.866397    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e4cc43ab-deda-4426-83f5-236f95bfdfc4-tls-certs\") pod \"kube-auth-proxy-75fdd59bb7-cpmzt\" (UID: \"e4cc43ab-deda-4426-83f5-236f95bfdfc4\") " pod="openshift-ingress/kube-auth-proxy-75fdd59bb7-cpmzt"
Oct 30 13:03:59.892470 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:03:59.892446    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vcccx\" (UniqueName: \"kubernetes.io/projected/e4cc43ab-deda-4426-83f5-236f95bfdfc4-kube-api-access-vcccx\") pod \"kube-auth-proxy-75fdd59bb7-cpmzt\" (UID: \"e4cc43ab-deda-4426-83f5-236f95bfdfc4\") " pod="openshift-ingress/kube-auth-proxy-75fdd59bb7-cpmzt"
Oct 30 13:04:00.061516 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:04:00.061445    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-75fdd59bb7-cpmzt"
Oct 30 13:04:00.360407 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:04:00.360372    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-75fdd59bb7-cpmzt"]
Oct 30 13:04:00.363407 ip-10-0-142-84 kubenswrapper[4943]: W1030 13:04:00.363382    4943 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode4cc43ab_deda_4426_83f5_236f95bfdfc4.slice/crio-661405ff5caed8b2c64091bcf619ec32f07cdc4a5fa0c0186e455234ac34564b WatchSource:0}: Error finding container 661405ff5caed8b2c64091bcf619ec32f07cdc4a5fa0c0186e455234ac34564b: Status 404 returned error can't find the container with id 661405ff5caed8b2c64091bcf619ec32f07cdc4a5fa0c0186e455234ac34564b
Oct 30 13:04:01.046834 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:04:01.046800    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-75fdd59bb7-cpmzt" event={"ID":"e4cc43ab-deda-4426-83f5-236f95bfdfc4","Type":"ContainerStarted","Data":"661405ff5caed8b2c64091bcf619ec32f07cdc4a5fa0c0186e455234ac34564b"}
Oct 30 13:04:01.554286 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:04:01.554240    4943 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7876bfcb74-qdqcg"]
Oct 30 13:04:01.557813 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:04:01.557789    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-qdqcg"
Oct 30 13:04:01.575943 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:04:01.575920    4943 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Oct 30 13:04:01.576280 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:04:01.576235    4943 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Oct 30 13:04:01.578800 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:04:01.578774    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/8b5abb19-7088-4c50-a2bc-403898cfc08d-local-certs\") pod \"istiod-openshift-gateway-7876bfcb74-qdqcg\" (UID: \"8b5abb19-7088-4c50-a2bc-403898cfc08d\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-qdqcg"
Oct 30 13:04:01.578904 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:04:01.578812    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/8b5abb19-7088-4c50-a2bc-403898cfc08d-istio-kubeconfig\") pod \"istiod-openshift-gateway-7876bfcb74-qdqcg\" (UID: \"8b5abb19-7088-4c50-a2bc-403898cfc08d\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-qdqcg"
Oct 30 13:04:01.578904 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:04:01.578897    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bdg7p\" (UniqueName: \"kubernetes.io/projected/8b5abb19-7088-4c50-a2bc-403898cfc08d-kube-api-access-bdg7p\") pod \"istiod-openshift-gateway-7876bfcb74-qdqcg\" (UID: \"8b5abb19-7088-4c50-a2bc-403898cfc08d\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-qdqcg"
Oct 30 13:04:01.579028 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:04:01.578972    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/8b5abb19-7088-4c50-a2bc-403898cfc08d-cacerts\") pod \"istiod-openshift-gateway-7876bfcb74-qdqcg\" (UID: \"8b5abb19-7088-4c50-a2bc-403898cfc08d\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-qdqcg"
Oct 30 13:04:01.579028 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:04:01.579017    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8b5abb19-7088-4c50-a2bc-403898cfc08d-istio-token\") pod \"istiod-openshift-gateway-7876bfcb74-qdqcg\" (UID: \"8b5abb19-7088-4c50-a2bc-403898cfc08d\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-qdqcg"
Oct 30 13:04:01.579128 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:04:01.579057    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/8b5abb19-7088-4c50-a2bc-403898cfc08d-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7876bfcb74-qdqcg\" (UID: \"8b5abb19-7088-4c50-a2bc-403898cfc08d\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-qdqcg"
Oct 30 13:04:01.579128 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:04:01.579083    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/8b5abb19-7088-4c50-a2bc-403898cfc08d-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7876bfcb74-qdqcg\" (UID: \"8b5abb19-7088-4c50-a2bc-403898cfc08d\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-qdqcg"
Oct 30 13:04:01.591107 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:04:01.589464    4943 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-6ngqs\""
Oct 30 13:04:01.591107 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:04:01.589506    4943 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Oct 30 13:04:01.591107 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:04:01.589534    4943 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Oct 30 13:04:01.591355 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:04:01.591128    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7876bfcb74-qdqcg"]
Oct 30 13:04:01.679992 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:04:01.679957    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/8b5abb19-7088-4c50-a2bc-403898cfc08d-cacerts\") pod \"istiod-openshift-gateway-7876bfcb74-qdqcg\" (UID: \"8b5abb19-7088-4c50-a2bc-403898cfc08d\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-qdqcg"
Oct 30 13:04:01.680155 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:04:01.680004    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8b5abb19-7088-4c50-a2bc-403898cfc08d-istio-token\") pod \"istiod-openshift-gateway-7876bfcb74-qdqcg\" (UID: \"8b5abb19-7088-4c50-a2bc-403898cfc08d\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-qdqcg"
Oct 30 13:04:01.680155 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:04:01.680024    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/8b5abb19-7088-4c50-a2bc-403898cfc08d-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7876bfcb74-qdqcg\" (UID: \"8b5abb19-7088-4c50-a2bc-403898cfc08d\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-qdqcg"
Oct 30 13:04:01.680155 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:04:01.680059    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/8b5abb19-7088-4c50-a2bc-403898cfc08d-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7876bfcb74-qdqcg\" (UID: \"8b5abb19-7088-4c50-a2bc-403898cfc08d\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-qdqcg"
Oct 30 13:04:01.680155 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:04:01.680117    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/8b5abb19-7088-4c50-a2bc-403898cfc08d-local-certs\") pod \"istiod-openshift-gateway-7876bfcb74-qdqcg\" (UID: \"8b5abb19-7088-4c50-a2bc-403898cfc08d\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-qdqcg"
Oct 30 13:04:01.684104 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:04:01.680161    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/8b5abb19-7088-4c50-a2bc-403898cfc08d-istio-kubeconfig\") pod \"istiod-openshift-gateway-7876bfcb74-qdqcg\" (UID: \"8b5abb19-7088-4c50-a2bc-403898cfc08d\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-qdqcg"
Oct 30 13:04:01.684104 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:04:01.680224    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bdg7p\" (UniqueName: \"kubernetes.io/projected/8b5abb19-7088-4c50-a2bc-403898cfc08d-kube-api-access-bdg7p\") pod \"istiod-openshift-gateway-7876bfcb74-qdqcg\" (UID: \"8b5abb19-7088-4c50-a2bc-403898cfc08d\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-qdqcg"
Oct 30 13:04:01.684104 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:04:01.681324    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/8b5abb19-7088-4c50-a2bc-403898cfc08d-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7876bfcb74-qdqcg\" (UID: \"8b5abb19-7088-4c50-a2bc-403898cfc08d\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-qdqcg"
Oct 30 13:04:01.684104 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:04:01.682779    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/8b5abb19-7088-4c50-a2bc-403898cfc08d-cacerts\") pod \"istiod-openshift-gateway-7876bfcb74-qdqcg\" (UID: \"8b5abb19-7088-4c50-a2bc-403898cfc08d\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-qdqcg"
Oct 30 13:04:01.684104 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:04:01.683517    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/8b5abb19-7088-4c50-a2bc-403898cfc08d-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7876bfcb74-qdqcg\" (UID: \"8b5abb19-7088-4c50-a2bc-403898cfc08d\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-qdqcg"
Oct 30 13:04:01.684104 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:04:01.683706    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/8b5abb19-7088-4c50-a2bc-403898cfc08d-istio-kubeconfig\") pod \"istiod-openshift-gateway-7876bfcb74-qdqcg\" (UID: \"8b5abb19-7088-4c50-a2bc-403898cfc08d\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-qdqcg"
Oct 30 13:04:01.684728 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:04:01.684688    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/8b5abb19-7088-4c50-a2bc-403898cfc08d-local-certs\") pod \"istiod-openshift-gateway-7876bfcb74-qdqcg\" (UID: \"8b5abb19-7088-4c50-a2bc-403898cfc08d\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-qdqcg"
Oct 30 13:04:01.706152 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:04:01.706130    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bdg7p\" (UniqueName: \"kubernetes.io/projected/8b5abb19-7088-4c50-a2bc-403898cfc08d-kube-api-access-bdg7p\") pod \"istiod-openshift-gateway-7876bfcb74-qdqcg\" (UID: \"8b5abb19-7088-4c50-a2bc-403898cfc08d\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-qdqcg"
Oct 30 13:04:01.706582 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:04:01.706559    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8b5abb19-7088-4c50-a2bc-403898cfc08d-istio-token\") pod \"istiod-openshift-gateway-7876bfcb74-qdqcg\" (UID: \"8b5abb19-7088-4c50-a2bc-403898cfc08d\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-qdqcg"
Oct 30 13:04:01.872907 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:04:01.872804    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-qdqcg"
Oct 30 13:04:02.224226 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:04:02.224196    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7876bfcb74-qdqcg"]
Oct 30 13:04:02.228636 ip-10-0-142-84 kubenswrapper[4943]: W1030 13:04:02.228603    4943 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8b5abb19_7088_4c50_a2bc_403898cfc08d.slice/crio-6d46dd378026f6b460fc32f09350577682ef9f1e4763b62256c9002bd73ac553 WatchSource:0}: Error finding container 6d46dd378026f6b460fc32f09350577682ef9f1e4763b62256c9002bd73ac553: Status 404 returned error can't find the container with id 6d46dd378026f6b460fc32f09350577682ef9f1e4763b62256c9002bd73ac553
Oct 30 13:04:03.059066 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:04:03.058981    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-75fdd59bb7-cpmzt" event={"ID":"e4cc43ab-deda-4426-83f5-236f95bfdfc4","Type":"ContainerStarted","Data":"2ecffbf2f5df569957ab85f0c51b2d6e342a5a64e1bd9b05f6fe39d9ebb5bd7c"}
Oct 30 13:04:03.060513 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:04:03.060479    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-qdqcg" event={"ID":"8b5abb19-7088-4c50-a2bc-403898cfc08d","Type":"ContainerStarted","Data":"6d46dd378026f6b460fc32f09350577682ef9f1e4763b62256c9002bd73ac553"}
Oct 30 13:04:03.105659 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:04:03.105600    4943 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-75fdd59bb7-cpmzt" podStartSLOduration=1.746082456 podStartE2EDuration="4.105581685s" totalImagesPullingTime="2.359499229s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:03:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:04:03.104256175 +0000 UTC m=+418.892259918" watchObservedRunningTime="2026-10-30 13:04:03.105581685 +0000 UTC m=+418.893585416"
Oct 30 13:04:04.477634 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:04:04.477602    4943 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 13:04:04.477923 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:04:04.477662    4943 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 13:04:05.073689 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:04:05.073649    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-qdqcg" event={"ID":"8b5abb19-7088-4c50-a2bc-403898cfc08d","Type":"ContainerStarted","Data":"643a5755d4413ea0ea6bcfee0a59eef8651a5cb004ca875620150201a772ae62"}
Oct 30 13:04:05.073893 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:04:05.073845    4943 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-qdqcg"
Oct 30 13:04:05.075744 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:04:05.075715    4943 patch_prober.go:29] interesting pod/istiod-openshift-gateway-7876bfcb74-qdqcg container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 13:04:05.075887 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:04:05.075770    4943 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-qdqcg" podUID="8b5abb19-7088-4c50-a2bc-403898cfc08d" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 13:04:05.164227 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:04:05.164173    4943 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-qdqcg" podStartSLOduration=1.918557917 podStartE2EDuration="4.164159915s" totalImagesPullingTime="2.245601998s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:04:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:04:05.149688692 +0000 UTC m=+420.937692421" watchObservedRunningTime="2026-10-30 13:04:05.164159915 +0000 UTC m=+420.952163645"
Oct 30 13:04:05.954622 ip-10-0-142-84 kubenswrapper[4943]: E1030 13:04:05.954582    4943 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-knqz6" containerName="extractor"
Oct 30 13:04:06.078424 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:04:06.078379    4943 patch_prober.go:29] interesting pod/istiod-openshift-gateway-7876bfcb74-qdqcg container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 13:04:06.078577 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:04:06.078440    4943 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-qdqcg" podUID="8b5abb19-7088-4c50-a2bc-403898cfc08d" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 13:04:07.026232 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:04:07.026192    4943 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-observability-584f99c788-mwqh2"
Oct 30 13:04:09.078900 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:04:09.078865    4943 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-qdqcg"
Oct 30 13:04:14.937070 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:04:14.937026    4943 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-module-controller-manager-5cfb77bf8c-8r9mm"]
Oct 30 13:04:14.942121 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:04:14.942100    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-module-controller-manager-5cfb77bf8c-8r9mm"
Oct 30 13:04:14.963877 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:04:14.963854    4943 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-module-controller-manager-dockercfg-bkx6m\""
Oct 30 13:04:14.993740 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:04:14.993710    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/6eb5db74-9830-41a6-8778-7e7e68213a8a-manifests\") pod \"kserve-module-controller-manager-5cfb77bf8c-8r9mm\" (UID: \"6eb5db74-9830-41a6-8778-7e7e68213a8a\") " pod="opendatahub/kserve-module-controller-manager-5cfb77bf8c-8r9mm"
Oct 30 13:04:14.993881 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:04:14.993758    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6eb5db74-9830-41a6-8778-7e7e68213a8a-tmp\") pod \"kserve-module-controller-manager-5cfb77bf8c-8r9mm\" (UID: \"6eb5db74-9830-41a6-8778-7e7e68213a8a\") " pod="opendatahub/kserve-module-controller-manager-5cfb77bf8c-8r9mm"
Oct 30 13:04:14.993881 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:04:14.993775    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/6eb5db74-9830-41a6-8778-7e7e68213a8a-trusted-ca\") pod \"kserve-module-controller-manager-5cfb77bf8c-8r9mm\" (UID: \"6eb5db74-9830-41a6-8778-7e7e68213a8a\") " pod="opendatahub/kserve-module-controller-manager-5cfb77bf8c-8r9mm"
Oct 30 13:04:14.993992 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:04:14.993880    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9lpfq\" (UniqueName: \"kubernetes.io/projected/6eb5db74-9830-41a6-8778-7e7e68213a8a-kube-api-access-9lpfq\") pod \"kserve-module-controller-manager-5cfb77bf8c-8r9mm\" (UID: \"6eb5db74-9830-41a6-8778-7e7e68213a8a\") " pod="opendatahub/kserve-module-controller-manager-5cfb77bf8c-8r9mm"
Oct 30 13:04:15.009342 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:04:15.008611    4943 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"odh-trusted-ca-bundle\""
Oct 30 13:04:15.009342 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:04:15.009159    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-module-controller-manager-5cfb77bf8c-8r9mm"]
Oct 30 13:04:15.094236 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:04:15.094199    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6eb5db74-9830-41a6-8778-7e7e68213a8a-tmp\") pod \"kserve-module-controller-manager-5cfb77bf8c-8r9mm\" (UID: \"6eb5db74-9830-41a6-8778-7e7e68213a8a\") " pod="opendatahub/kserve-module-controller-manager-5cfb77bf8c-8r9mm"
Oct 30 13:04:15.094236 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:04:15.094234    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/6eb5db74-9830-41a6-8778-7e7e68213a8a-trusted-ca\") pod \"kserve-module-controller-manager-5cfb77bf8c-8r9mm\" (UID: \"6eb5db74-9830-41a6-8778-7e7e68213a8a\") " pod="opendatahub/kserve-module-controller-manager-5cfb77bf8c-8r9mm"
Oct 30 13:04:15.094460 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:04:15.094253    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9lpfq\" (UniqueName: \"kubernetes.io/projected/6eb5db74-9830-41a6-8778-7e7e68213a8a-kube-api-access-9lpfq\") pod \"kserve-module-controller-manager-5cfb77bf8c-8r9mm\" (UID: \"6eb5db74-9830-41a6-8778-7e7e68213a8a\") " pod="opendatahub/kserve-module-controller-manager-5cfb77bf8c-8r9mm"
Oct 30 13:04:15.094460 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:04:15.094386    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/6eb5db74-9830-41a6-8778-7e7e68213a8a-manifests\") pod \"kserve-module-controller-manager-5cfb77bf8c-8r9mm\" (UID: \"6eb5db74-9830-41a6-8778-7e7e68213a8a\") " pod="opendatahub/kserve-module-controller-manager-5cfb77bf8c-8r9mm"
Oct 30 13:04:15.094619 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:04:15.094597    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6eb5db74-9830-41a6-8778-7e7e68213a8a-tmp\") pod \"kserve-module-controller-manager-5cfb77bf8c-8r9mm\" (UID: \"6eb5db74-9830-41a6-8778-7e7e68213a8a\") " pod="opendatahub/kserve-module-controller-manager-5cfb77bf8c-8r9mm"
Oct 30 13:04:15.094698 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:04:15.094679    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/6eb5db74-9830-41a6-8778-7e7e68213a8a-manifests\") pod \"kserve-module-controller-manager-5cfb77bf8c-8r9mm\" (UID: \"6eb5db74-9830-41a6-8778-7e7e68213a8a\") " pod="opendatahub/kserve-module-controller-manager-5cfb77bf8c-8r9mm"
Oct 30 13:04:15.095091 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:04:15.095075    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/6eb5db74-9830-41a6-8778-7e7e68213a8a-trusted-ca\") pod \"kserve-module-controller-manager-5cfb77bf8c-8r9mm\" (UID: \"6eb5db74-9830-41a6-8778-7e7e68213a8a\") " pod="opendatahub/kserve-module-controller-manager-5cfb77bf8c-8r9mm"
Oct 30 13:04:15.124174 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:04:15.124147    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9lpfq\" (UniqueName: \"kubernetes.io/projected/6eb5db74-9830-41a6-8778-7e7e68213a8a-kube-api-access-9lpfq\") pod \"kserve-module-controller-manager-5cfb77bf8c-8r9mm\" (UID: \"6eb5db74-9830-41a6-8778-7e7e68213a8a\") " pod="opendatahub/kserve-module-controller-manager-5cfb77bf8c-8r9mm"
Oct 30 13:04:15.254085 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:04:15.254001    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-module-controller-manager-5cfb77bf8c-8r9mm"
Oct 30 13:04:15.575125 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:04:15.575093    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-module-controller-manager-5cfb77bf8c-8r9mm"]
Oct 30 13:04:15.576968 ip-10-0-142-84 kubenswrapper[4943]: W1030 13:04:15.576929    4943 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6eb5db74_9830_41a6_8778_7e7e68213a8a.slice/crio-d9e09af8d815da5d58943cc958f7d6f84fab872be32c294ad128aafd32ed9e61 WatchSource:0}: Error finding container d9e09af8d815da5d58943cc958f7d6f84fab872be32c294ad128aafd32ed9e61: Status 404 returned error can't find the container with id d9e09af8d815da5d58943cc958f7d6f84fab872be32c294ad128aafd32ed9e61
Oct 30 13:04:16.123053 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:04:16.123014    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-module-controller-manager-5cfb77bf8c-8r9mm" event={"ID":"6eb5db74-9830-41a6-8778-7e7e68213a8a","Type":"ContainerStarted","Data":"d9e09af8d815da5d58943cc958f7d6f84fab872be32c294ad128aafd32ed9e61"}
Oct 30 13:04:17.129690 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:04:17.129649    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-module-controller-manager-5cfb77bf8c-8r9mm" event={"ID":"6eb5db74-9830-41a6-8778-7e7e68213a8a","Type":"ContainerStarted","Data":"150cd851cbcd5f717ecce4f95fadfd198bc318dcc2741e23af5764e9ed66fb40"}
Oct 30 13:04:17.129973 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:04:17.129784    4943 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-module-controller-manager-5cfb77bf8c-8r9mm"
Oct 30 13:04:17.265434 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:04:17.265366    4943 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/kserve-module-controller-manager-5cfb77bf8c-8r9mm" podStartSLOduration=1.783004738 podStartE2EDuration="3.265347444s" totalImagesPullingTime="1.482342706s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:04:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:04:17.264563626 +0000 UTC m=+433.052567360" watchObservedRunningTime="2026-10-30 13:04:17.265347444 +0000 UTC m=+433.053351176"
Oct 30 13:04:35.955014 ip-10-0-142-84 kubenswrapper[4943]: E1030 13:04:35.954978    4943 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-knqz6" containerName="extractor"
Oct 30 13:04:38.135194 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:04:38.135163    4943 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-module-controller-manager-5cfb77bf8c-8r9mm"
Oct 30 13:05:03.614355 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:05:03.614318    4943 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-vvkvh"]
Oct 30 13:05:03.622425 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:05:03.622402    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-vvkvh"
Oct 30 13:05:03.662988 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:05:03.662960    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-vvkvh"]
Oct 30 13:05:03.685784 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:05:03.685758    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/1d6ac649-474a-4e72-b202-259df21727a5-istio-kubeconfig\") pod \"istiod-openshift-gateway-7ddd9cfdc8-vvkvh\" (UID: \"1d6ac649-474a-4e72-b202-259df21727a5\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-vvkvh"
Oct 30 13:05:03.685921 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:05:03.685797    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/1d6ac649-474a-4e72-b202-259df21727a5-cacerts\") pod \"istiod-openshift-gateway-7ddd9cfdc8-vvkvh\" (UID: \"1d6ac649-474a-4e72-b202-259df21727a5\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-vvkvh"
Oct 30 13:05:03.685921 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:05:03.685833    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/1d6ac649-474a-4e72-b202-259df21727a5-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7ddd9cfdc8-vvkvh\" (UID: \"1d6ac649-474a-4e72-b202-259df21727a5\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-vvkvh"
Oct 30 13:05:03.686016 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:05:03.685978    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1d6ac649-474a-4e72-b202-259df21727a5-istio-token\") pod \"istiod-openshift-gateway-7ddd9cfdc8-vvkvh\" (UID: \"1d6ac649-474a-4e72-b202-259df21727a5\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-vvkvh"
Oct 30 13:05:03.686064 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:05:03.686047    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/1d6ac649-474a-4e72-b202-259df21727a5-local-certs\") pod \"istiod-openshift-gateway-7ddd9cfdc8-vvkvh\" (UID: \"1d6ac649-474a-4e72-b202-259df21727a5\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-vvkvh"
Oct 30 13:05:03.686119 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:05:03.686092    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/1d6ac649-474a-4e72-b202-259df21727a5-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7ddd9cfdc8-vvkvh\" (UID: \"1d6ac649-474a-4e72-b202-259df21727a5\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-vvkvh"
Oct 30 13:05:03.686171 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:05:03.686148    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kd2p8\" (UniqueName: \"kubernetes.io/projected/1d6ac649-474a-4e72-b202-259df21727a5-kube-api-access-kd2p8\") pod \"istiod-openshift-gateway-7ddd9cfdc8-vvkvh\" (UID: \"1d6ac649-474a-4e72-b202-259df21727a5\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-vvkvh"
Oct 30 13:05:03.787344 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:05:03.787310    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kd2p8\" (UniqueName: \"kubernetes.io/projected/1d6ac649-474a-4e72-b202-259df21727a5-kube-api-access-kd2p8\") pod \"istiod-openshift-gateway-7ddd9cfdc8-vvkvh\" (UID: \"1d6ac649-474a-4e72-b202-259df21727a5\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-vvkvh"
Oct 30 13:05:03.787505 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:05:03.787377    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/1d6ac649-474a-4e72-b202-259df21727a5-istio-kubeconfig\") pod \"istiod-openshift-gateway-7ddd9cfdc8-vvkvh\" (UID: \"1d6ac649-474a-4e72-b202-259df21727a5\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-vvkvh"
Oct 30 13:05:03.787505 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:05:03.787464    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/1d6ac649-474a-4e72-b202-259df21727a5-cacerts\") pod \"istiod-openshift-gateway-7ddd9cfdc8-vvkvh\" (UID: \"1d6ac649-474a-4e72-b202-259df21727a5\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-vvkvh"
Oct 30 13:05:03.787505 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:05:03.787487    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/1d6ac649-474a-4e72-b202-259df21727a5-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7ddd9cfdc8-vvkvh\" (UID: \"1d6ac649-474a-4e72-b202-259df21727a5\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-vvkvh"
Oct 30 13:05:03.787505 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:05:03.787505    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1d6ac649-474a-4e72-b202-259df21727a5-istio-token\") pod \"istiod-openshift-gateway-7ddd9cfdc8-vvkvh\" (UID: \"1d6ac649-474a-4e72-b202-259df21727a5\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-vvkvh"
Oct 30 13:05:03.787725 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:05:03.787651    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/1d6ac649-474a-4e72-b202-259df21727a5-local-certs\") pod \"istiod-openshift-gateway-7ddd9cfdc8-vvkvh\" (UID: \"1d6ac649-474a-4e72-b202-259df21727a5\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-vvkvh"
Oct 30 13:05:03.787725 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:05:03.787704    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/1d6ac649-474a-4e72-b202-259df21727a5-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7ddd9cfdc8-vvkvh\" (UID: \"1d6ac649-474a-4e72-b202-259df21727a5\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-vvkvh"
Oct 30 13:05:03.788255 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:05:03.788223    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/1d6ac649-474a-4e72-b202-259df21727a5-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7ddd9cfdc8-vvkvh\" (UID: \"1d6ac649-474a-4e72-b202-259df21727a5\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-vvkvh"
Oct 30 13:05:03.789967 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:05:03.789928    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/1d6ac649-474a-4e72-b202-259df21727a5-local-certs\") pod \"istiod-openshift-gateway-7ddd9cfdc8-vvkvh\" (UID: \"1d6ac649-474a-4e72-b202-259df21727a5\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-vvkvh"
Oct 30 13:05:03.790079 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:05:03.790037    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/1d6ac649-474a-4e72-b202-259df21727a5-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7ddd9cfdc8-vvkvh\" (UID: \"1d6ac649-474a-4e72-b202-259df21727a5\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-vvkvh"
Oct 30 13:05:03.790227 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:05:03.790208    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/1d6ac649-474a-4e72-b202-259df21727a5-cacerts\") pod \"istiod-openshift-gateway-7ddd9cfdc8-vvkvh\" (UID: \"1d6ac649-474a-4e72-b202-259df21727a5\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-vvkvh"
Oct 30 13:05:03.790316 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:05:03.790285    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/1d6ac649-474a-4e72-b202-259df21727a5-istio-kubeconfig\") pod \"istiod-openshift-gateway-7ddd9cfdc8-vvkvh\" (UID: \"1d6ac649-474a-4e72-b202-259df21727a5\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-vvkvh"
Oct 30 13:05:03.818917 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:05:03.818893    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1d6ac649-474a-4e72-b202-259df21727a5-istio-token\") pod \"istiod-openshift-gateway-7ddd9cfdc8-vvkvh\" (UID: \"1d6ac649-474a-4e72-b202-259df21727a5\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-vvkvh"
Oct 30 13:05:03.819734 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:05:03.819714    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kd2p8\" (UniqueName: \"kubernetes.io/projected/1d6ac649-474a-4e72-b202-259df21727a5-kube-api-access-kd2p8\") pod \"istiod-openshift-gateway-7ddd9cfdc8-vvkvh\" (UID: \"1d6ac649-474a-4e72-b202-259df21727a5\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-vvkvh"
Oct 30 13:05:03.934868 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:05:03.934843    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-vvkvh"
Oct 30 13:05:04.278032 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:05:04.277944    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-vvkvh"]
Oct 30 13:05:04.293555 ip-10-0-142-84 kubenswrapper[4943]: W1030 13:05:04.293527    4943 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1d6ac649_474a_4e72_b202_259df21727a5.slice/crio-2af4cc3064850cfcee5f0628c28c39ae47f7e1adb3558d7702a6233a08a75046 WatchSource:0}: Error finding container 2af4cc3064850cfcee5f0628c28c39ae47f7e1adb3558d7702a6233a08a75046: Status 404 returned error can't find the container with id 2af4cc3064850cfcee5f0628c28c39ae47f7e1adb3558d7702a6233a08a75046
Oct 30 13:05:04.295486 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:05:04.295453    4943 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 13:05:04.295569 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:05:04.295517    4943 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 13:05:04.341701 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:05:04.341671    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-vvkvh" event={"ID":"1d6ac649-474a-4e72-b202-259df21727a5","Type":"ContainerStarted","Data":"2af4cc3064850cfcee5f0628c28c39ae47f7e1adb3558d7702a6233a08a75046"}
Oct 30 13:05:05.347694 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:05:05.347653    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-vvkvh" event={"ID":"1d6ac649-474a-4e72-b202-259df21727a5","Type":"ContainerStarted","Data":"03d9057d31248fd8ccd8e390e88ef2574ea59cd9d71c76285ea9a514f7841b38"}
Oct 30 13:05:05.348160 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:05:05.347709    4943 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-vvkvh"
Oct 30 13:05:05.349693 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:05:05.349666    4943 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-vvkvh"
Oct 30 13:05:05.497630 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:05:05.497576    4943 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-vvkvh" podStartSLOduration=2.497560854 podStartE2EDuration="2.497560854s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:05:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:05:05.483662685 +0000 UTC m=+481.271666417" watchObservedRunningTime="2026-10-30 13:05:05.497560854 +0000 UTC m=+481.285564578"
Oct 30 13:05:05.606984 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:05:05.606898    4943 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7876bfcb74-qdqcg"]
Oct 30 13:05:05.607155 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:05:05.607129    4943 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-qdqcg" podUID="8b5abb19-7088-4c50-a2bc-403898cfc08d" containerName="discovery" containerID="cri-o://643a5755d4413ea0ea6bcfee0a59eef8651a5cb004ca875620150201a772ae62" gracePeriod=30
Oct 30 13:05:05.855368 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:05:05.855341    4943 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-qdqcg"
Oct 30 13:05:05.905202 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:05:05.905135    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/8b5abb19-7088-4c50-a2bc-403898cfc08d-istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/8b5abb19-7088-4c50-a2bc-403898cfc08d-istio-csr-dns-cert\") pod \"8b5abb19-7088-4c50-a2bc-403898cfc08d\" (UID: \"8b5abb19-7088-4c50-a2bc-403898cfc08d\") "
Oct 30 13:05:05.905202 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:05:05.905174    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8b5abb19-7088-4c50-a2bc-403898cfc08d-kube-api-access-bdg7p\" (UniqueName: \"kubernetes.io/projected/8b5abb19-7088-4c50-a2bc-403898cfc08d-kube-api-access-bdg7p\") pod \"8b5abb19-7088-4c50-a2bc-403898cfc08d\" (UID: \"8b5abb19-7088-4c50-a2bc-403898cfc08d\") "
Oct 30 13:05:05.905202 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:05:05.905193    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8b5abb19-7088-4c50-a2bc-403898cfc08d-istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/8b5abb19-7088-4c50-a2bc-403898cfc08d-istio-csr-ca-configmap\") pod \"8b5abb19-7088-4c50-a2bc-403898cfc08d\" (UID: \"8b5abb19-7088-4c50-a2bc-403898cfc08d\") "
Oct 30 13:05:05.905457 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:05:05.905222    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8b5abb19-7088-4c50-a2bc-403898cfc08d-istio-token\" (UniqueName: \"kubernetes.io/projected/8b5abb19-7088-4c50-a2bc-403898cfc08d-istio-token\") pod \"8b5abb19-7088-4c50-a2bc-403898cfc08d\" (UID: \"8b5abb19-7088-4c50-a2bc-403898cfc08d\") "
Oct 30 13:05:05.905457 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:05:05.905283    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/8b5abb19-7088-4c50-a2bc-403898cfc08d-istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/8b5abb19-7088-4c50-a2bc-403898cfc08d-istio-kubeconfig\") pod \"8b5abb19-7088-4c50-a2bc-403898cfc08d\" (UID: \"8b5abb19-7088-4c50-a2bc-403898cfc08d\") "
Oct 30 13:05:05.905457 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:05:05.905343    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8b5abb19-7088-4c50-a2bc-403898cfc08d-local-certs\" (UniqueName: \"kubernetes.io/empty-dir/8b5abb19-7088-4c50-a2bc-403898cfc08d-local-certs\") pod \"8b5abb19-7088-4c50-a2bc-403898cfc08d\" (UID: \"8b5abb19-7088-4c50-a2bc-403898cfc08d\") "
Oct 30 13:05:05.905457 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:05:05.905367    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/8b5abb19-7088-4c50-a2bc-403898cfc08d-cacerts\" (UniqueName: \"kubernetes.io/secret/8b5abb19-7088-4c50-a2bc-403898cfc08d-cacerts\") pod \"8b5abb19-7088-4c50-a2bc-403898cfc08d\" (UID: \"8b5abb19-7088-4c50-a2bc-403898cfc08d\") "
Oct 30 13:05:05.905673 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:05:05.905607    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8b5abb19-7088-4c50-a2bc-403898cfc08d-istio-csr-ca-configmap" pod "8b5abb19-7088-4c50-a2bc-403898cfc08d" (UID: "8b5abb19-7088-4c50-a2bc-403898cfc08d"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 13:05:05.907770 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:05:05.907743    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8b5abb19-7088-4c50-a2bc-403898cfc08d-istio-kubeconfig" pod "8b5abb19-7088-4c50-a2bc-403898cfc08d" (UID: "8b5abb19-7088-4c50-a2bc-403898cfc08d"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 13:05:05.907985 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:05:05.907961    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8b5abb19-7088-4c50-a2bc-403898cfc08d-cacerts" pod "8b5abb19-7088-4c50-a2bc-403898cfc08d" (UID: "8b5abb19-7088-4c50-a2bc-403898cfc08d"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 13:05:05.908063 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:05:05.907963    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8b5abb19-7088-4c50-a2bc-403898cfc08d-istio-token" pod "8b5abb19-7088-4c50-a2bc-403898cfc08d" (UID: "8b5abb19-7088-4c50-a2bc-403898cfc08d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:05:05.908063 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:05:05.908007    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8b5abb19-7088-4c50-a2bc-403898cfc08d-kube-api-access-bdg7p" pod "8b5abb19-7088-4c50-a2bc-403898cfc08d" (UID: "8b5abb19-7088-4c50-a2bc-403898cfc08d"). InnerVolumeSpecName "kube-api-access-bdg7p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:05:05.908150 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:05:05.908063    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8b5abb19-7088-4c50-a2bc-403898cfc08d-local-certs" pod "8b5abb19-7088-4c50-a2bc-403898cfc08d" (UID: "8b5abb19-7088-4c50-a2bc-403898cfc08d"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:05:05.908150 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:05:05.908122    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8b5abb19-7088-4c50-a2bc-403898cfc08d-istio-csr-dns-cert" pod "8b5abb19-7088-4c50-a2bc-403898cfc08d" (UID: "8b5abb19-7088-4c50-a2bc-403898cfc08d"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 13:05:05.957924 ip-10-0-142-84 kubenswrapper[4943]: E1030 13:05:05.957874    4943 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-knqz6" containerName="extractor"
Oct 30 13:05:06.006593 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:05:06.006547    4943 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/8b5abb19-7088-4c50-a2bc-403898cfc08d-istio-csr-dns-cert\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:05:06.006593 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:05:06.006587    4943 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bdg7p\" (UniqueName: \"kubernetes.io/projected/8b5abb19-7088-4c50-a2bc-403898cfc08d-kube-api-access-bdg7p\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:05:06.006593 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:05:06.006597    4943 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/8b5abb19-7088-4c50-a2bc-403898cfc08d-istio-csr-ca-configmap\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:05:06.006593 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:05:06.006607    4943 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8b5abb19-7088-4c50-a2bc-403898cfc08d-istio-token\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:05:06.006839 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:05:06.006615    4943 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/8b5abb19-7088-4c50-a2bc-403898cfc08d-istio-kubeconfig\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:05:06.006839 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:05:06.006623    4943 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/8b5abb19-7088-4c50-a2bc-403898cfc08d-local-certs\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:05:06.006839 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:05:06.006631    4943 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/8b5abb19-7088-4c50-a2bc-403898cfc08d-cacerts\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:05:06.353434 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:05:06.353405    4943 generic.go:356] "Generic (PLEG): container finished" podID="8b5abb19-7088-4c50-a2bc-403898cfc08d" containerID="643a5755d4413ea0ea6bcfee0a59eef8651a5cb004ca875620150201a772ae62" exitCode=0
Oct 30 13:05:06.353846 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:05:06.353473    4943 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-qdqcg"
Oct 30 13:05:06.353846 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:05:06.353492    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-qdqcg" event={"ID":"8b5abb19-7088-4c50-a2bc-403898cfc08d","Type":"ContainerDied","Data":"643a5755d4413ea0ea6bcfee0a59eef8651a5cb004ca875620150201a772ae62"}
Oct 30 13:05:06.353846 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:05:06.353540    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-qdqcg" event={"ID":"8b5abb19-7088-4c50-a2bc-403898cfc08d","Type":"ContainerDied","Data":"6d46dd378026f6b460fc32f09350577682ef9f1e4763b62256c9002bd73ac553"}
Oct 30 13:05:06.353846 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:05:06.353559    4943 scope.go:122] "RemoveContainer" containerID="643a5755d4413ea0ea6bcfee0a59eef8651a5cb004ca875620150201a772ae62"
Oct 30 13:05:06.363007 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:05:06.362990    4943 scope.go:122] "RemoveContainer" containerID="643a5755d4413ea0ea6bcfee0a59eef8651a5cb004ca875620150201a772ae62"
Oct 30 13:05:06.363312 ip-10-0-142-84 kubenswrapper[4943]: E1030 13:05:06.363238    4943 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"643a5755d4413ea0ea6bcfee0a59eef8651a5cb004ca875620150201a772ae62\": container with ID starting with 643a5755d4413ea0ea6bcfee0a59eef8651a5cb004ca875620150201a772ae62 not found: ID does not exist" containerID="643a5755d4413ea0ea6bcfee0a59eef8651a5cb004ca875620150201a772ae62"
Oct 30 13:05:06.363312 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:05:06.363282    4943 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"643a5755d4413ea0ea6bcfee0a59eef8651a5cb004ca875620150201a772ae62"} err="failed to get container status \"643a5755d4413ea0ea6bcfee0a59eef8651a5cb004ca875620150201a772ae62\": rpc error: code = NotFound desc = could not find container \"643a5755d4413ea0ea6bcfee0a59eef8651a5cb004ca875620150201a772ae62\": container with ID starting with 643a5755d4413ea0ea6bcfee0a59eef8651a5cb004ca875620150201a772ae62 not found: ID does not exist"
Oct 30 13:05:06.410780 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:05:06.410745    4943 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7876bfcb74-qdqcg"]
Oct 30 13:05:06.414738 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:05:06.414714    4943 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7876bfcb74-qdqcg"]
Oct 30 13:05:08.348670 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:05:08.348636    4943 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8b5abb19-7088-4c50-a2bc-403898cfc08d" path="/var/lib/kubelet/pods/8b5abb19-7088-4c50-a2bc-403898cfc08d/volumes"
Oct 30 13:05:35.954597 ip-10-0-142-84 kubenswrapper[4943]: E1030 13:05:35.954512    4943 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-knqz6" containerName="extractor"
Oct 30 13:06:05.954065 ip-10-0-142-84 kubenswrapper[4943]: E1030 13:06:05.954023    4943 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-knqz6" containerName="extractor"
Oct 30 13:06:19.011539 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:06:19.011505    4943 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c69hpcj"]
Oct 30 13:06:19.011933 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:06:19.011856    4943 cpu_manager.go:414] "containerMap: removing container" podUID="8b5abb19-7088-4c50-a2bc-403898cfc08d" containerName="discovery"
Oct 30 13:06:19.011933 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:06:19.011868    4943 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8b5abb19-7088-4c50-a2bc-403898cfc08d" containerName="discovery"
Oct 30 13:06:19.012020 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:06:19.011947    4943 memory_manager.go:365] "RemoveStaleState removing state" podUID="8b5abb19-7088-4c50-a2bc-403898cfc08d" containerName="discovery"
Oct 30 13:06:19.015157 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:06:19.015135    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c69hpcj"
Oct 30 13:06:19.048494 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:06:19.048458    4943 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Oct 30 13:06:19.048494 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:06:19.048459    4943 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3-b83c8cba-dockercfg-cncmp\""
Oct 30 13:06:19.048688 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:06:19.048459    4943 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Oct 30 13:06:19.049939 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:06:19.049915    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c69hpcj"]
Oct 30 13:06:19.077454 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:06:19.077429    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/fe6b9420-1813-4212-9afb-15b3e9f616f1-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c69hpcj\" (UID: \"fe6b9420-1813-4212-9afb-15b3e9f616f1\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c69hpcj"
Oct 30 13:06:19.077554 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:06:19.077478    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r5gj4\" (UniqueName: \"kubernetes.io/projected/fe6b9420-1813-4212-9afb-15b3e9f616f1-kube-api-access-r5gj4\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c69hpcj\" (UID: \"fe6b9420-1813-4212-9afb-15b3e9f616f1\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c69hpcj"
Oct 30 13:06:19.077602 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:06:19.077563    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/fe6b9420-1813-4212-9afb-15b3e9f616f1-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c69hpcj\" (UID: \"fe6b9420-1813-4212-9afb-15b3e9f616f1\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c69hpcj"
Oct 30 13:06:19.109621 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:06:19.109598    4943 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ajkfrv"]
Oct 30 13:06:19.113136 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:06:19.113119    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ajkfrv"
Oct 30 13:06:19.134847 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:06:19.134825    4943 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"6541d93aaee335d6227a8778ab4e840bbe7f52-f98fbcf8-dockercfg-lsk6g\""
Oct 30 13:06:19.136581 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:06:19.136558    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ajkfrv"]
Oct 30 13:06:19.178718 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:06:19.178687    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n7bsj\" (UniqueName: \"kubernetes.io/projected/17f5a8d2-67e0-4489-8734-00f0d4483788-kube-api-access-n7bsj\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ajkfrv\" (UID: \"17f5a8d2-67e0-4489-8734-00f0d4483788\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ajkfrv"
Oct 30 13:06:19.178819 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:06:19.178729    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/17f5a8d2-67e0-4489-8734-00f0d4483788-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ajkfrv\" (UID: \"17f5a8d2-67e0-4489-8734-00f0d4483788\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ajkfrv"
Oct 30 13:06:19.178819 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:06:19.178757    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/fe6b9420-1813-4212-9afb-15b3e9f616f1-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c69hpcj\" (UID: \"fe6b9420-1813-4212-9afb-15b3e9f616f1\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c69hpcj"
Oct 30 13:06:19.178904 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:06:19.178833    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r5gj4\" (UniqueName: \"kubernetes.io/projected/fe6b9420-1813-4212-9afb-15b3e9f616f1-kube-api-access-r5gj4\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c69hpcj\" (UID: \"fe6b9420-1813-4212-9afb-15b3e9f616f1\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c69hpcj"
Oct 30 13:06:19.178904 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:06:19.178880    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/fe6b9420-1813-4212-9afb-15b3e9f616f1-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c69hpcj\" (UID: \"fe6b9420-1813-4212-9afb-15b3e9f616f1\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c69hpcj"
Oct 30 13:06:19.178904 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:06:19.178900    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/17f5a8d2-67e0-4489-8734-00f0d4483788-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ajkfrv\" (UID: \"17f5a8d2-67e0-4489-8734-00f0d4483788\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ajkfrv"
Oct 30 13:06:19.179196 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:06:19.179179    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/fe6b9420-1813-4212-9afb-15b3e9f616f1-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c69hpcj\" (UID: \"fe6b9420-1813-4212-9afb-15b3e9f616f1\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c69hpcj"
Oct 30 13:06:19.179231 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:06:19.179204    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/fe6b9420-1813-4212-9afb-15b3e9f616f1-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c69hpcj\" (UID: \"fe6b9420-1813-4212-9afb-15b3e9f616f1\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c69hpcj"
Oct 30 13:06:19.205136 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:06:19.205112    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r5gj4\" (UniqueName: \"kubernetes.io/projected/fe6b9420-1813-4212-9afb-15b3e9f616f1-kube-api-access-r5gj4\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c69hpcj\" (UID: \"fe6b9420-1813-4212-9afb-15b3e9f616f1\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c69hpcj"
Oct 30 13:06:19.215720 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:06:19.215699    4943 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7mn9zh"]
Oct 30 13:06:19.219400 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:06:19.219384    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7mn9zh"
Oct 30 13:06:19.240111 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:06:19.240090    4943 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"da112bd609bff87f7bacd2ba941225d7cff15c-bd73b2fc-dockercfg-m6r2l\""
Oct 30 13:06:19.243173 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:06:19.243151    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7mn9zh"]
Oct 30 13:06:19.280202 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:06:19.280142    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/17f5a8d2-67e0-4489-8734-00f0d4483788-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ajkfrv\" (UID: \"17f5a8d2-67e0-4489-8734-00f0d4483788\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ajkfrv"
Oct 30 13:06:19.280339 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:06:19.280198    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/25c7a9c5-16a6-4206-9071-a2c437b8318d-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7mn9zh\" (UID: \"25c7a9c5-16a6-4206-9071-a2c437b8318d\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7mn9zh"
Oct 30 13:06:19.280339 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:06:19.280242    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bs2b6\" (UniqueName: \"kubernetes.io/projected/25c7a9c5-16a6-4206-9071-a2c437b8318d-kube-api-access-bs2b6\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7mn9zh\" (UID: \"25c7a9c5-16a6-4206-9071-a2c437b8318d\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7mn9zh"
Oct 30 13:06:19.280417 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:06:19.280347    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n7bsj\" (UniqueName: \"kubernetes.io/projected/17f5a8d2-67e0-4489-8734-00f0d4483788-kube-api-access-n7bsj\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ajkfrv\" (UID: \"17f5a8d2-67e0-4489-8734-00f0d4483788\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ajkfrv"
Oct 30 13:06:19.280417 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:06:19.280392    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/17f5a8d2-67e0-4489-8734-00f0d4483788-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ajkfrv\" (UID: \"17f5a8d2-67e0-4489-8734-00f0d4483788\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ajkfrv"
Oct 30 13:06:19.280502 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:06:19.280436    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/25c7a9c5-16a6-4206-9071-a2c437b8318d-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7mn9zh\" (UID: \"25c7a9c5-16a6-4206-9071-a2c437b8318d\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7mn9zh"
Oct 30 13:06:19.280583 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:06:19.280566    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/17f5a8d2-67e0-4489-8734-00f0d4483788-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ajkfrv\" (UID: \"17f5a8d2-67e0-4489-8734-00f0d4483788\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ajkfrv"
Oct 30 13:06:19.280750 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:06:19.280731    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/17f5a8d2-67e0-4489-8734-00f0d4483788-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ajkfrv\" (UID: \"17f5a8d2-67e0-4489-8734-00f0d4483788\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ajkfrv"
Oct 30 13:06:19.306695 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:06:19.306671    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n7bsj\" (UniqueName: \"kubernetes.io/projected/17f5a8d2-67e0-4489-8734-00f0d4483788-kube-api-access-n7bsj\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ajkfrv\" (UID: \"17f5a8d2-67e0-4489-8734-00f0d4483788\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ajkfrv"
Oct 30 13:06:19.323473 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:06:19.323448    4943 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cgwwks"]
Oct 30 13:06:19.326957 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:06:19.326942    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cgwwks"
Oct 30 13:06:19.330804 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:06:19.330789    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c69hpcj"
Oct 30 13:06:19.343681 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:06:19.343662    4943 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"e3d011e69d9573cabb8c359e5ac709eb5ac240-be78d160-dockercfg-7pv5m\""
Oct 30 13:06:19.348835 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:06:19.348813    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cgwwks"]
Oct 30 13:06:19.381481 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:06:19.381440    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6rskw\" (UniqueName: \"kubernetes.io/projected/214211da-eba8-4117-b03e-da53414fbfce-kube-api-access-6rskw\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cgwwks\" (UID: \"214211da-eba8-4117-b03e-da53414fbfce\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cgwwks"
Oct 30 13:06:19.381661 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:06:19.381503    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/214211da-eba8-4117-b03e-da53414fbfce-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cgwwks\" (UID: \"214211da-eba8-4117-b03e-da53414fbfce\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cgwwks"
Oct 30 13:06:19.381661 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:06:19.381549    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/25c7a9c5-16a6-4206-9071-a2c437b8318d-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7mn9zh\" (UID: \"25c7a9c5-16a6-4206-9071-a2c437b8318d\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7mn9zh"
Oct 30 13:06:19.381661 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:06:19.381599    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bs2b6\" (UniqueName: \"kubernetes.io/projected/25c7a9c5-16a6-4206-9071-a2c437b8318d-kube-api-access-bs2b6\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7mn9zh\" (UID: \"25c7a9c5-16a6-4206-9071-a2c437b8318d\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7mn9zh"
Oct 30 13:06:19.381661 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:06:19.381653    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/25c7a9c5-16a6-4206-9071-a2c437b8318d-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7mn9zh\" (UID: \"25c7a9c5-16a6-4206-9071-a2c437b8318d\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7mn9zh"
Oct 30 13:06:19.381891 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:06:19.381684    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/214211da-eba8-4117-b03e-da53414fbfce-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cgwwks\" (UID: \"214211da-eba8-4117-b03e-da53414fbfce\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cgwwks"
Oct 30 13:06:19.383122 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:06:19.382230    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/25c7a9c5-16a6-4206-9071-a2c437b8318d-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7mn9zh\" (UID: \"25c7a9c5-16a6-4206-9071-a2c437b8318d\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7mn9zh"
Oct 30 13:06:19.383122 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:06:19.382767    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/25c7a9c5-16a6-4206-9071-a2c437b8318d-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7mn9zh\" (UID: \"25c7a9c5-16a6-4206-9071-a2c437b8318d\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7mn9zh"
Oct 30 13:06:19.408543 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:06:19.408255    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bs2b6\" (UniqueName: \"kubernetes.io/projected/25c7a9c5-16a6-4206-9071-a2c437b8318d-kube-api-access-bs2b6\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7mn9zh\" (UID: \"25c7a9c5-16a6-4206-9071-a2c437b8318d\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7mn9zh"
Oct 30 13:06:19.428096 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:06:19.428070    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ajkfrv"
Oct 30 13:06:19.483287 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:06:19.482510    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6rskw\" (UniqueName: \"kubernetes.io/projected/214211da-eba8-4117-b03e-da53414fbfce-kube-api-access-6rskw\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cgwwks\" (UID: \"214211da-eba8-4117-b03e-da53414fbfce\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cgwwks"
Oct 30 13:06:19.483287 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:06:19.482563    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/214211da-eba8-4117-b03e-da53414fbfce-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cgwwks\" (UID: \"214211da-eba8-4117-b03e-da53414fbfce\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cgwwks"
Oct 30 13:06:19.483287 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:06:19.482674    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/214211da-eba8-4117-b03e-da53414fbfce-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cgwwks\" (UID: \"214211da-eba8-4117-b03e-da53414fbfce\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cgwwks"
Oct 30 13:06:19.483763 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:06:19.483736    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/214211da-eba8-4117-b03e-da53414fbfce-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cgwwks\" (UID: \"214211da-eba8-4117-b03e-da53414fbfce\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cgwwks"
Oct 30 13:06:19.484960 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:06:19.484929    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/214211da-eba8-4117-b03e-da53414fbfce-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cgwwks\" (UID: \"214211da-eba8-4117-b03e-da53414fbfce\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cgwwks"
Oct 30 13:06:19.512282 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:06:19.508681    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6rskw\" (UniqueName: \"kubernetes.io/projected/214211da-eba8-4117-b03e-da53414fbfce-kube-api-access-6rskw\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cgwwks\" (UID: \"214211da-eba8-4117-b03e-da53414fbfce\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cgwwks"
Oct 30 13:06:19.535821 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:06:19.535772    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7mn9zh"
Oct 30 13:06:19.639405 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:06:19.639382    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c69hpcj"]
Oct 30 13:06:19.641325 ip-10-0-142-84 kubenswrapper[4943]: W1030 13:06:19.641299    4943 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfe6b9420_1813_4212_9afb_15b3e9f616f1.slice/crio-78d0e88a09d93af808a51b895c4fc6bd9762d8924570dd7dee219cf106a89302 WatchSource:0}: Error finding container 78d0e88a09d93af808a51b895c4fc6bd9762d8924570dd7dee219cf106a89302: Status 404 returned error can't find the container with id 78d0e88a09d93af808a51b895c4fc6bd9762d8924570dd7dee219cf106a89302
Oct 30 13:06:19.644274 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:06:19.644236    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cgwwks"
Oct 30 13:06:19.666617 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:06:19.666594    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c69hpcj" event={"ID":"fe6b9420-1813-4212-9afb-15b3e9f616f1","Type":"ContainerStarted","Data":"78d0e88a09d93af808a51b895c4fc6bd9762d8924570dd7dee219cf106a89302"}
Oct 30 13:06:19.727860 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:06:19.727833    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ajkfrv"]
Oct 30 13:06:19.730899 ip-10-0-142-84 kubenswrapper[4943]: W1030 13:06:19.730872    4943 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod17f5a8d2_67e0_4489_8734_00f0d4483788.slice/crio-d147532ce44b1ec1a7d9e7fc0efc7752fe86bc6953afd822a4613d0b95f41bda WatchSource:0}: Error finding container d147532ce44b1ec1a7d9e7fc0efc7752fe86bc6953afd822a4613d0b95f41bda: Status 404 returned error can't find the container with id d147532ce44b1ec1a7d9e7fc0efc7752fe86bc6953afd822a4613d0b95f41bda
Oct 30 13:06:19.837013 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:06:19.836987    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7mn9zh"]
Oct 30 13:06:19.838502 ip-10-0-142-84 kubenswrapper[4943]: W1030 13:06:19.838479    4943 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod25c7a9c5_16a6_4206_9071_a2c437b8318d.slice/crio-24812a05d74b8b76da3e3932ab6e93ce33dd8ca5107807ad5eea0ff6c743c895 WatchSource:0}: Error finding container 24812a05d74b8b76da3e3932ab6e93ce33dd8ca5107807ad5eea0ff6c743c895: Status 404 returned error can't find the container with id 24812a05d74b8b76da3e3932ab6e93ce33dd8ca5107807ad5eea0ff6c743c895
Oct 30 13:06:19.951074 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:06:19.951050    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cgwwks"]
Oct 30 13:06:20.008203 ip-10-0-142-84 kubenswrapper[4943]: W1030 13:06:20.008172    4943 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod214211da_eba8_4117_b03e_da53414fbfce.slice/crio-fadfa5053772061aebcb678b813d45499f46bab48aaa73a1735efa62b6fc6d6a WatchSource:0}: Error finding container fadfa5053772061aebcb678b813d45499f46bab48aaa73a1735efa62b6fc6d6a: Status 404 returned error can't find the container with id fadfa5053772061aebcb678b813d45499f46bab48aaa73a1735efa62b6fc6d6a
Oct 30 13:06:20.673756 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:06:20.673722    4943 generic.go:356] "Generic (PLEG): container finished" podID="214211da-eba8-4117-b03e-da53414fbfce" containerID="2a6207c68cdc6fe8e0a0cb14735fba5ae69e28eb2e88e990bb98f51752855935" exitCode=0
Oct 30 13:06:20.674230 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:06:20.673801    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cgwwks" event={"ID":"214211da-eba8-4117-b03e-da53414fbfce","Type":"ContainerDied","Data":"2a6207c68cdc6fe8e0a0cb14735fba5ae69e28eb2e88e990bb98f51752855935"}
Oct 30 13:06:20.674380 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:06:20.674282    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cgwwks" event={"ID":"214211da-eba8-4117-b03e-da53414fbfce","Type":"ContainerStarted","Data":"fadfa5053772061aebcb678b813d45499f46bab48aaa73a1735efa62b6fc6d6a"}
Oct 30 13:06:20.675896 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:06:20.675870    4943 generic.go:356] "Generic (PLEG): container finished" podID="25c7a9c5-16a6-4206-9071-a2c437b8318d" containerID="f0e588b7fddbe9424caba16980ff0df34003be484e73f65aaee3f290263f6d82" exitCode=0
Oct 30 13:06:20.676009 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:06:20.675903    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7mn9zh" event={"ID":"25c7a9c5-16a6-4206-9071-a2c437b8318d","Type":"ContainerDied","Data":"f0e588b7fddbe9424caba16980ff0df34003be484e73f65aaee3f290263f6d82"}
Oct 30 13:06:20.676009 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:06:20.675948    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7mn9zh" event={"ID":"25c7a9c5-16a6-4206-9071-a2c437b8318d","Type":"ContainerStarted","Data":"24812a05d74b8b76da3e3932ab6e93ce33dd8ca5107807ad5eea0ff6c743c895"}
Oct 30 13:06:20.677755 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:06:20.677720    4943 generic.go:356] "Generic (PLEG): container finished" podID="17f5a8d2-67e0-4489-8734-00f0d4483788" containerID="9458252ea5700fe9691641347431ef57b85932f3e0d7368b0dbf817798dafbdb" exitCode=0
Oct 30 13:06:20.677907 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:06:20.677805    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ajkfrv" event={"ID":"17f5a8d2-67e0-4489-8734-00f0d4483788","Type":"ContainerDied","Data":"9458252ea5700fe9691641347431ef57b85932f3e0d7368b0dbf817798dafbdb"}
Oct 30 13:06:20.677907 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:06:20.677840    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ajkfrv" event={"ID":"17f5a8d2-67e0-4489-8734-00f0d4483788","Type":"ContainerStarted","Data":"d147532ce44b1ec1a7d9e7fc0efc7752fe86bc6953afd822a4613d0b95f41bda"}
Oct 30 13:06:20.679573 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:06:20.679524    4943 generic.go:356] "Generic (PLEG): container finished" podID="fe6b9420-1813-4212-9afb-15b3e9f616f1" containerID="22269ea506d6e970ec78c4921e5c43e97cf17a1a7fd22323e10ca9d7111b4450" exitCode=0
Oct 30 13:06:20.679646 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:06:20.679572    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c69hpcj" event={"ID":"fe6b9420-1813-4212-9afb-15b3e9f616f1","Type":"ContainerDied","Data":"22269ea506d6e970ec78c4921e5c43e97cf17a1a7fd22323e10ca9d7111b4450"}
Oct 30 13:06:21.685722 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:06:21.685692    4943 generic.go:356] "Generic (PLEG): container finished" podID="25c7a9c5-16a6-4206-9071-a2c437b8318d" containerID="9161a565d782c062517635b0c8a363d3d29563ced06249099436da810a563419" exitCode=0
Oct 30 13:06:21.686146 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:06:21.685778    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7mn9zh" event={"ID":"25c7a9c5-16a6-4206-9071-a2c437b8318d","Type":"ContainerDied","Data":"9161a565d782c062517635b0c8a363d3d29563ced06249099436da810a563419"}
Oct 30 13:06:21.687845 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:06:21.687820    4943 generic.go:356] "Generic (PLEG): container finished" podID="214211da-eba8-4117-b03e-da53414fbfce" containerID="2162a7ce9c9c0ec2c44ee744ecd8cbb0958d7e50c350bc30d3d255a6439c332c" exitCode=0
Oct 30 13:06:21.687939 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:06:21.687900    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cgwwks" event={"ID":"214211da-eba8-4117-b03e-da53414fbfce","Type":"ContainerDied","Data":"2162a7ce9c9c0ec2c44ee744ecd8cbb0958d7e50c350bc30d3d255a6439c332c"}
Oct 30 13:06:22.694864 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:06:22.694827    4943 generic.go:356] "Generic (PLEG): container finished" podID="25c7a9c5-16a6-4206-9071-a2c437b8318d" containerID="a354da6fdefbe2080d8b635a71cfe3695002e327e6c8d68f2bbccbec9941e70b" exitCode=0
Oct 30 13:06:22.695307 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:06:22.694903    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7mn9zh" event={"ID":"25c7a9c5-16a6-4206-9071-a2c437b8318d","Type":"ContainerDied","Data":"a354da6fdefbe2080d8b635a71cfe3695002e327e6c8d68f2bbccbec9941e70b"}
Oct 30 13:06:22.696611 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:06:22.696588    4943 generic.go:356] "Generic (PLEG): container finished" podID="fe6b9420-1813-4212-9afb-15b3e9f616f1" containerID="5abe181328fb7e0fb4a8eb948a1184277983e4118ae57103885dfc8b3833e560" exitCode=0
Oct 30 13:06:22.696719 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:06:22.696660    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c69hpcj" event={"ID":"fe6b9420-1813-4212-9afb-15b3e9f616f1","Type":"ContainerDied","Data":"5abe181328fb7e0fb4a8eb948a1184277983e4118ae57103885dfc8b3833e560"}
Oct 30 13:06:22.699094 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:06:22.699069    4943 generic.go:356] "Generic (PLEG): container finished" podID="214211da-eba8-4117-b03e-da53414fbfce" containerID="fea0576998608d897f7ff2ea9a30a911d131f93bd1995c9036a545ae31566957" exitCode=0
Oct 30 13:06:22.699176 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:06:22.699152    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cgwwks" event={"ID":"214211da-eba8-4117-b03e-da53414fbfce","Type":"ContainerDied","Data":"fea0576998608d897f7ff2ea9a30a911d131f93bd1995c9036a545ae31566957"}
Oct 30 13:06:23.706177 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:06:23.706138    4943 generic.go:356] "Generic (PLEG): container finished" podID="fe6b9420-1813-4212-9afb-15b3e9f616f1" containerID="10d356aea9dc268fdeb719d889679e94d8e4cd7a157b92b40add28267f64b56d" exitCode=0
Oct 30 13:06:23.706667 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:06:23.706216    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c69hpcj" event={"ID":"fe6b9420-1813-4212-9afb-15b3e9f616f1","Type":"ContainerDied","Data":"10d356aea9dc268fdeb719d889679e94d8e4cd7a157b92b40add28267f64b56d"}
Oct 30 13:06:23.845406 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:06:23.845373    4943 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7mn9zh"
Oct 30 13:06:23.874861 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:06:23.874834    4943 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cgwwks"
Oct 30 13:06:23.920826 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:06:23.920802    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/25c7a9c5-16a6-4206-9071-a2c437b8318d-kube-api-access-bs2b6\" (UniqueName: \"kubernetes.io/projected/25c7a9c5-16a6-4206-9071-a2c437b8318d-kube-api-access-bs2b6\") pod \"25c7a9c5-16a6-4206-9071-a2c437b8318d\" (UID: \"25c7a9c5-16a6-4206-9071-a2c437b8318d\") "
Oct 30 13:06:23.920976 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:06:23.920892    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/25c7a9c5-16a6-4206-9071-a2c437b8318d-bundle\" (UniqueName: \"kubernetes.io/empty-dir/25c7a9c5-16a6-4206-9071-a2c437b8318d-bundle\") pod \"25c7a9c5-16a6-4206-9071-a2c437b8318d\" (UID: \"25c7a9c5-16a6-4206-9071-a2c437b8318d\") "
Oct 30 13:06:23.920976 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:06:23.920956    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/25c7a9c5-16a6-4206-9071-a2c437b8318d-util\" (UniqueName: \"kubernetes.io/empty-dir/25c7a9c5-16a6-4206-9071-a2c437b8318d-util\") pod \"25c7a9c5-16a6-4206-9071-a2c437b8318d\" (UID: \"25c7a9c5-16a6-4206-9071-a2c437b8318d\") "
Oct 30 13:06:23.921339 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:06:23.921312    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/25c7a9c5-16a6-4206-9071-a2c437b8318d-bundle" pod "25c7a9c5-16a6-4206-9071-a2c437b8318d" (UID: "25c7a9c5-16a6-4206-9071-a2c437b8318d"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:06:23.922952 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:06:23.922929    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/25c7a9c5-16a6-4206-9071-a2c437b8318d-kube-api-access-bs2b6" pod "25c7a9c5-16a6-4206-9071-a2c437b8318d" (UID: "25c7a9c5-16a6-4206-9071-a2c437b8318d"). InnerVolumeSpecName "kube-api-access-bs2b6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:06:23.926559 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:06:23.926530    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/25c7a9c5-16a6-4206-9071-a2c437b8318d-util" pod "25c7a9c5-16a6-4206-9071-a2c437b8318d" (UID: "25c7a9c5-16a6-4206-9071-a2c437b8318d"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:06:24.022225 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:06:24.022149    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/214211da-eba8-4117-b03e-da53414fbfce-bundle\" (UniqueName: \"kubernetes.io/empty-dir/214211da-eba8-4117-b03e-da53414fbfce-bundle\") pod \"214211da-eba8-4117-b03e-da53414fbfce\" (UID: \"214211da-eba8-4117-b03e-da53414fbfce\") "
Oct 30 13:06:24.022225 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:06:24.022206    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/214211da-eba8-4117-b03e-da53414fbfce-util\" (UniqueName: \"kubernetes.io/empty-dir/214211da-eba8-4117-b03e-da53414fbfce-util\") pod \"214211da-eba8-4117-b03e-da53414fbfce\" (UID: \"214211da-eba8-4117-b03e-da53414fbfce\") "
Oct 30 13:06:24.022457 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:06:24.022246    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/214211da-eba8-4117-b03e-da53414fbfce-kube-api-access-6rskw\" (UniqueName: \"kubernetes.io/projected/214211da-eba8-4117-b03e-da53414fbfce-kube-api-access-6rskw\") pod \"214211da-eba8-4117-b03e-da53414fbfce\" (UID: \"214211da-eba8-4117-b03e-da53414fbfce\") "
Oct 30 13:06:24.022457 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:06:24.022402    4943 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/25c7a9c5-16a6-4206-9071-a2c437b8318d-util\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:06:24.022457 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:06:24.022414    4943 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bs2b6\" (UniqueName: \"kubernetes.io/projected/25c7a9c5-16a6-4206-9071-a2c437b8318d-kube-api-access-bs2b6\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:06:24.022457 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:06:24.022423    4943 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/25c7a9c5-16a6-4206-9071-a2c437b8318d-bundle\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:06:24.022783 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:06:24.022755    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/214211da-eba8-4117-b03e-da53414fbfce-bundle" pod "214211da-eba8-4117-b03e-da53414fbfce" (UID: "214211da-eba8-4117-b03e-da53414fbfce"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:06:24.024432 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:06:24.024410    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/214211da-eba8-4117-b03e-da53414fbfce-kube-api-access-6rskw" pod "214211da-eba8-4117-b03e-da53414fbfce" (UID: "214211da-eba8-4117-b03e-da53414fbfce"). InnerVolumeSpecName "kube-api-access-6rskw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:06:24.027387 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:06:24.027365    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/214211da-eba8-4117-b03e-da53414fbfce-util" pod "214211da-eba8-4117-b03e-da53414fbfce" (UID: "214211da-eba8-4117-b03e-da53414fbfce"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:06:24.123377 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:06:24.123353    4943 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6rskw\" (UniqueName: \"kubernetes.io/projected/214211da-eba8-4117-b03e-da53414fbfce-kube-api-access-6rskw\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:06:24.123377 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:06:24.123377    4943 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/214211da-eba8-4117-b03e-da53414fbfce-bundle\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:06:24.123516 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:06:24.123389    4943 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/214211da-eba8-4117-b03e-da53414fbfce-util\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:06:24.713106 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:06:24.713073    4943 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cgwwks"
Oct 30 13:06:24.713106 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:06:24.713095    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cgwwks" event={"ID":"214211da-eba8-4117-b03e-da53414fbfce","Type":"ContainerDied","Data":"fadfa5053772061aebcb678b813d45499f46bab48aaa73a1735efa62b6fc6d6a"}
Oct 30 13:06:24.713586 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:06:24.713141    4943 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="fadfa5053772061aebcb678b813d45499f46bab48aaa73a1735efa62b6fc6d6a"
Oct 30 13:06:24.715228 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:06:24.715198    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7mn9zh" event={"ID":"25c7a9c5-16a6-4206-9071-a2c437b8318d","Type":"ContainerDied","Data":"24812a05d74b8b76da3e3932ab6e93ce33dd8ca5107807ad5eea0ff6c743c895"}
Oct 30 13:06:24.715228 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:06:24.715218    4943 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7mn9zh"
Oct 30 13:06:24.715439 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:06:24.715248    4943 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="24812a05d74b8b76da3e3932ab6e93ce33dd8ca5107807ad5eea0ff6c743c895"
Oct 30 13:06:24.845685 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:06:24.845661    4943 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c69hpcj"
Oct 30 13:06:24.928856 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:06:24.928819    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fe6b9420-1813-4212-9afb-15b3e9f616f1-util\" (UniqueName: \"kubernetes.io/empty-dir/fe6b9420-1813-4212-9afb-15b3e9f616f1-util\") pod \"fe6b9420-1813-4212-9afb-15b3e9f616f1\" (UID: \"fe6b9420-1813-4212-9afb-15b3e9f616f1\") "
Oct 30 13:06:24.929004 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:06:24.928875    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fe6b9420-1813-4212-9afb-15b3e9f616f1-kube-api-access-r5gj4\" (UniqueName: \"kubernetes.io/projected/fe6b9420-1813-4212-9afb-15b3e9f616f1-kube-api-access-r5gj4\") pod \"fe6b9420-1813-4212-9afb-15b3e9f616f1\" (UID: \"fe6b9420-1813-4212-9afb-15b3e9f616f1\") "
Oct 30 13:06:24.929004 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:06:24.928926    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fe6b9420-1813-4212-9afb-15b3e9f616f1-bundle\" (UniqueName: \"kubernetes.io/empty-dir/fe6b9420-1813-4212-9afb-15b3e9f616f1-bundle\") pod \"fe6b9420-1813-4212-9afb-15b3e9f616f1\" (UID: \"fe6b9420-1813-4212-9afb-15b3e9f616f1\") "
Oct 30 13:06:24.929436 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:06:24.929403    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fe6b9420-1813-4212-9afb-15b3e9f616f1-bundle" pod "fe6b9420-1813-4212-9afb-15b3e9f616f1" (UID: "fe6b9420-1813-4212-9afb-15b3e9f616f1"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:06:24.931071 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:06:24.931045    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fe6b9420-1813-4212-9afb-15b3e9f616f1-kube-api-access-r5gj4" pod "fe6b9420-1813-4212-9afb-15b3e9f616f1" (UID: "fe6b9420-1813-4212-9afb-15b3e9f616f1"). InnerVolumeSpecName "kube-api-access-r5gj4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:06:24.934411 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:06:24.934385    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fe6b9420-1813-4212-9afb-15b3e9f616f1-util" pod "fe6b9420-1813-4212-9afb-15b3e9f616f1" (UID: "fe6b9420-1813-4212-9afb-15b3e9f616f1"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:06:25.029488 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:06:25.029424    4943 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/fe6b9420-1813-4212-9afb-15b3e9f616f1-util\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:06:25.029488 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:06:25.029448    4943 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r5gj4\" (UniqueName: \"kubernetes.io/projected/fe6b9420-1813-4212-9afb-15b3e9f616f1-kube-api-access-r5gj4\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:06:25.029488 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:06:25.029460    4943 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/fe6b9420-1813-4212-9afb-15b3e9f616f1-bundle\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:06:25.722226 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:06:25.722190    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c69hpcj" event={"ID":"fe6b9420-1813-4212-9afb-15b3e9f616f1","Type":"ContainerDied","Data":"78d0e88a09d93af808a51b895c4fc6bd9762d8924570dd7dee219cf106a89302"}
Oct 30 13:06:25.722605 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:06:25.722239    4943 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="78d0e88a09d93af808a51b895c4fc6bd9762d8924570dd7dee219cf106a89302"
Oct 30 13:06:25.722605 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:06:25.722202    4943 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c69hpcj"
Oct 30 13:06:35.954464 ip-10-0-142-84 kubenswrapper[4943]: E1030 13:06:35.954361    4943 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-knqz6" containerName="extractor"
Oct 30 13:06:38.783819 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:06:38.783788    4943 generic.go:356] "Generic (PLEG): container finished" podID="17f5a8d2-67e0-4489-8734-00f0d4483788" containerID="d12be025a9aed04fb121238ca00aa05945385d2f08e187e6c35a99fee8af36c8" exitCode=0
Oct 30 13:06:38.784220 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:06:38.783878    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ajkfrv" event={"ID":"17f5a8d2-67e0-4489-8734-00f0d4483788","Type":"ContainerDied","Data":"d12be025a9aed04fb121238ca00aa05945385d2f08e187e6c35a99fee8af36c8"}
Oct 30 13:06:39.790414 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:06:39.790372    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ajkfrv" event={"ID":"17f5a8d2-67e0-4489-8734-00f0d4483788","Type":"ContainerStarted","Data":"7faa33ff90edd8bc5e2bb2fbd986919881632ed036097b21e141bf71bbfff456"}
Oct 30 13:06:39.840068 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:06:39.840017    4943 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ajkfrv" podStartSLOduration=3.274532516 podStartE2EDuration="20.840004747s" totalImagesPullingTime="17.565472231s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:06:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:06:39.838217098 +0000 UTC m=+575.626220807" watchObservedRunningTime="2026-10-30 13:06:39.840004747 +0000 UTC m=+575.628008487"
Oct 30 13:06:49.836137 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:06:49.836107    4943 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ajkfrv_17f5a8d2-67e0-4489-8734-00f0d4483788/extract/0.log"
Oct 30 13:06:49.836807 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:06:49.836784    4943 generic.go:356] "Generic (PLEG): container finished" podID="17f5a8d2-67e0-4489-8734-00f0d4483788" containerID="7faa33ff90edd8bc5e2bb2fbd986919881632ed036097b21e141bf71bbfff456" exitCode=1
Oct 30 13:06:49.836897 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:06:49.836861    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ajkfrv" event={"ID":"17f5a8d2-67e0-4489-8734-00f0d4483788","Type":"ContainerDied","Data":"7faa33ff90edd8bc5e2bb2fbd986919881632ed036097b21e141bf71bbfff456"}
Oct 30 13:06:50.964971 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:06:50.964949    4943 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ajkfrv_17f5a8d2-67e0-4489-8734-00f0d4483788/extract/0.log"
Oct 30 13:06:50.968119 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:06:50.968100    4943 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ajkfrv"
Oct 30 13:06:51.047991 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:06:51.047957    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/17f5a8d2-67e0-4489-8734-00f0d4483788-bundle\" (UniqueName: \"kubernetes.io/empty-dir/17f5a8d2-67e0-4489-8734-00f0d4483788-bundle\") pod \"17f5a8d2-67e0-4489-8734-00f0d4483788\" (UID: \"17f5a8d2-67e0-4489-8734-00f0d4483788\") "
Oct 30 13:06:51.048115 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:06:51.048083    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/17f5a8d2-67e0-4489-8734-00f0d4483788-kube-api-access-n7bsj\" (UniqueName: \"kubernetes.io/projected/17f5a8d2-67e0-4489-8734-00f0d4483788-kube-api-access-n7bsj\") pod \"17f5a8d2-67e0-4489-8734-00f0d4483788\" (UID: \"17f5a8d2-67e0-4489-8734-00f0d4483788\") "
Oct 30 13:06:51.048173 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:06:51.048141    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/17f5a8d2-67e0-4489-8734-00f0d4483788-util\" (UniqueName: \"kubernetes.io/empty-dir/17f5a8d2-67e0-4489-8734-00f0d4483788-util\") pod \"17f5a8d2-67e0-4489-8734-00f0d4483788\" (UID: \"17f5a8d2-67e0-4489-8734-00f0d4483788\") "
Oct 30 13:06:51.048849 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:06:51.048814    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/17f5a8d2-67e0-4489-8734-00f0d4483788-bundle" pod "17f5a8d2-67e0-4489-8734-00f0d4483788" (UID: "17f5a8d2-67e0-4489-8734-00f0d4483788"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:06:51.050301 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:06:51.050278    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/17f5a8d2-67e0-4489-8734-00f0d4483788-kube-api-access-n7bsj" pod "17f5a8d2-67e0-4489-8734-00f0d4483788" (UID: "17f5a8d2-67e0-4489-8734-00f0d4483788"). InnerVolumeSpecName "kube-api-access-n7bsj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:06:51.052906 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:06:51.052884    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/17f5a8d2-67e0-4489-8734-00f0d4483788-util" pod "17f5a8d2-67e0-4489-8734-00f0d4483788" (UID: "17f5a8d2-67e0-4489-8734-00f0d4483788"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:06:51.149469 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:06:51.149407    4943 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/17f5a8d2-67e0-4489-8734-00f0d4483788-util\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:06:51.149469 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:06:51.149439    4943 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/17f5a8d2-67e0-4489-8734-00f0d4483788-bundle\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:06:51.149469 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:06:51.149448    4943 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n7bsj\" (UniqueName: \"kubernetes.io/projected/17f5a8d2-67e0-4489-8734-00f0d4483788-kube-api-access-n7bsj\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:06:51.847232 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:06:51.847203    4943 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ajkfrv_17f5a8d2-67e0-4489-8734-00f0d4483788/extract/0.log"
Oct 30 13:06:51.847958 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:06:51.847931    4943 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ajkfrv"
Oct 30 13:06:51.848110 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:06:51.847954    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ajkfrv" event={"ID":"17f5a8d2-67e0-4489-8734-00f0d4483788","Type":"ContainerDied","Data":"d147532ce44b1ec1a7d9e7fc0efc7752fe86bc6953afd822a4613d0b95f41bda"}
Oct 30 13:06:51.848110 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:06:51.847997    4943 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d147532ce44b1ec1a7d9e7fc0efc7752fe86bc6953afd822a4613d0b95f41bda"
Oct 30 13:06:58.034412 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:06:58.034375    4943 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847apns8f"]
Oct 30 13:06:58.034792 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:06:58.034728    4943 cpu_manager.go:414] "containerMap: removing container" podUID="214211da-eba8-4117-b03e-da53414fbfce" containerName="extract"
Oct 30 13:06:58.034792 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:06:58.034741    4943 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="214211da-eba8-4117-b03e-da53414fbfce" containerName="extract"
Oct 30 13:06:58.034792 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:06:58.034752    4943 cpu_manager.go:414] "containerMap: removing container" podUID="17f5a8d2-67e0-4489-8734-00f0d4483788" containerName="pull"
Oct 30 13:06:58.034792 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:06:58.034757    4943 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="17f5a8d2-67e0-4489-8734-00f0d4483788" containerName="pull"
Oct 30 13:06:58.034792 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:06:58.034765    4943 cpu_manager.go:414] "containerMap: removing container" podUID="17f5a8d2-67e0-4489-8734-00f0d4483788" containerName="extract"
Oct 30 13:06:58.034792 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:06:58.034769    4943 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="17f5a8d2-67e0-4489-8734-00f0d4483788" containerName="extract"
Oct 30 13:06:58.034792 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:06:58.034795    4943 cpu_manager.go:414] "containerMap: removing container" podUID="214211da-eba8-4117-b03e-da53414fbfce" containerName="util"
Oct 30 13:06:58.034998 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:06:58.034800    4943 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="214211da-eba8-4117-b03e-da53414fbfce" containerName="util"
Oct 30 13:06:58.034998 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:06:58.034809    4943 cpu_manager.go:414] "containerMap: removing container" podUID="214211da-eba8-4117-b03e-da53414fbfce" containerName="pull"
Oct 30 13:06:58.034998 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:06:58.034813    4943 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="214211da-eba8-4117-b03e-da53414fbfce" containerName="pull"
Oct 30 13:06:58.034998 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:06:58.034819    4943 cpu_manager.go:414] "containerMap: removing container" podUID="fe6b9420-1813-4212-9afb-15b3e9f616f1" containerName="pull"
Oct 30 13:06:58.034998 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:06:58.034823    4943 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fe6b9420-1813-4212-9afb-15b3e9f616f1" containerName="pull"
Oct 30 13:06:58.034998 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:06:58.034829    4943 cpu_manager.go:414] "containerMap: removing container" podUID="fe6b9420-1813-4212-9afb-15b3e9f616f1" containerName="extract"
Oct 30 13:06:58.034998 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:06:58.034833    4943 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fe6b9420-1813-4212-9afb-15b3e9f616f1" containerName="extract"
Oct 30 13:06:58.034998 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:06:58.034839    4943 cpu_manager.go:414] "containerMap: removing container" podUID="fe6b9420-1813-4212-9afb-15b3e9f616f1" containerName="util"
Oct 30 13:06:58.034998 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:06:58.034843    4943 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fe6b9420-1813-4212-9afb-15b3e9f616f1" containerName="util"
Oct 30 13:06:58.034998 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:06:58.034850    4943 cpu_manager.go:414] "containerMap: removing container" podUID="25c7a9c5-16a6-4206-9071-a2c437b8318d" containerName="pull"
Oct 30 13:06:58.034998 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:06:58.034856    4943 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="25c7a9c5-16a6-4206-9071-a2c437b8318d" containerName="pull"
Oct 30 13:06:58.034998 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:06:58.034865    4943 cpu_manager.go:414] "containerMap: removing container" podUID="25c7a9c5-16a6-4206-9071-a2c437b8318d" containerName="extract"
Oct 30 13:06:58.034998 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:06:58.034872    4943 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="25c7a9c5-16a6-4206-9071-a2c437b8318d" containerName="extract"
Oct 30 13:06:58.034998 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:06:58.034893    4943 cpu_manager.go:414] "containerMap: removing container" podUID="17f5a8d2-67e0-4489-8734-00f0d4483788" containerName="util"
Oct 30 13:06:58.034998 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:06:58.034897    4943 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="17f5a8d2-67e0-4489-8734-00f0d4483788" containerName="util"
Oct 30 13:06:58.034998 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:06:58.034908    4943 cpu_manager.go:414] "containerMap: removing container" podUID="25c7a9c5-16a6-4206-9071-a2c437b8318d" containerName="util"
Oct 30 13:06:58.034998 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:06:58.034912    4943 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="25c7a9c5-16a6-4206-9071-a2c437b8318d" containerName="util"
Oct 30 13:06:58.034998 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:06:58.034974    4943 memory_manager.go:365] "RemoveStaleState removing state" podUID="214211da-eba8-4117-b03e-da53414fbfce" containerName="extract"
Oct 30 13:06:58.034998 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:06:58.034982    4943 memory_manager.go:365] "RemoveStaleState removing state" podUID="17f5a8d2-67e0-4489-8734-00f0d4483788" containerName="extract"
Oct 30 13:06:58.034998 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:06:58.034992    4943 memory_manager.go:365] "RemoveStaleState removing state" podUID="fe6b9420-1813-4212-9afb-15b3e9f616f1" containerName="extract"
Oct 30 13:06:58.034998 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:06:58.034997    4943 memory_manager.go:365] "RemoveStaleState removing state" podUID="25c7a9c5-16a6-4206-9071-a2c437b8318d" containerName="extract"
Oct 30 13:06:58.039888 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:06:58.039868    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847apns8f"
Oct 30 13:06:58.071731 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:06:58.071696    4943 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Oct 30 13:06:58.071866 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:06:58.071844    4943 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Oct 30 13:06:58.071959 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:06:58.071944    4943 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"6541d93aaee335d6227a8778ab4e840bbe7f52-f98fbcf8-dockercfg-lsk6g\""
Oct 30 13:06:58.073814 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:06:58.073792    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847apns8f"]
Oct 30 13:06:58.102782 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:06:58.102752    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/762613ee-244b-4c66-baf9-903b074059bf-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847apns8f\" (UID: \"762613ee-244b-4c66-baf9-903b074059bf\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847apns8f"
Oct 30 13:06:58.102879 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:06:58.102828    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7tp46\" (UniqueName: \"kubernetes.io/projected/762613ee-244b-4c66-baf9-903b074059bf-kube-api-access-7tp46\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847apns8f\" (UID: \"762613ee-244b-4c66-baf9-903b074059bf\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847apns8f"
Oct 30 13:06:58.102967 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:06:58.102949    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/762613ee-244b-4c66-baf9-903b074059bf-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847apns8f\" (UID: \"762613ee-244b-4c66-baf9-903b074059bf\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847apns8f"
Oct 30 13:06:58.203329 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:06:58.203300    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/762613ee-244b-4c66-baf9-903b074059bf-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847apns8f\" (UID: \"762613ee-244b-4c66-baf9-903b074059bf\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847apns8f"
Oct 30 13:06:58.203437 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:06:58.203360    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7tp46\" (UniqueName: \"kubernetes.io/projected/762613ee-244b-4c66-baf9-903b074059bf-kube-api-access-7tp46\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847apns8f\" (UID: \"762613ee-244b-4c66-baf9-903b074059bf\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847apns8f"
Oct 30 13:06:58.203437 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:06:58.203410    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/762613ee-244b-4c66-baf9-903b074059bf-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847apns8f\" (UID: \"762613ee-244b-4c66-baf9-903b074059bf\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847apns8f"
Oct 30 13:06:58.203670 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:06:58.203652    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/762613ee-244b-4c66-baf9-903b074059bf-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847apns8f\" (UID: \"762613ee-244b-4c66-baf9-903b074059bf\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847apns8f"
Oct 30 13:06:58.203709 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:06:58.203691    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/762613ee-244b-4c66-baf9-903b074059bf-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847apns8f\" (UID: \"762613ee-244b-4c66-baf9-903b074059bf\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847apns8f"
Oct 30 13:06:58.229138 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:06:58.229119    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7tp46\" (UniqueName: \"kubernetes.io/projected/762613ee-244b-4c66-baf9-903b074059bf-kube-api-access-7tp46\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847apns8f\" (UID: \"762613ee-244b-4c66-baf9-903b074059bf\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847apns8f"
Oct 30 13:06:58.356583 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:06:58.356512    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847apns8f"
Oct 30 13:06:58.662082 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:06:58.662055    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847apns8f"]
Oct 30 13:06:58.663667 ip-10-0-142-84 kubenswrapper[4943]: W1030 13:06:58.663628    4943 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod762613ee_244b_4c66_baf9_903b074059bf.slice/crio-3318a6feb035248926ba064aa7757a8a8958c9e4cd0d0ced72f5f4d8416c45cb WatchSource:0}: Error finding container 3318a6feb035248926ba064aa7757a8a8958c9e4cd0d0ced72f5f4d8416c45cb: Status 404 returned error can't find the container with id 3318a6feb035248926ba064aa7757a8a8958c9e4cd0d0ced72f5f4d8416c45cb
Oct 30 13:06:58.886784 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:06:58.886751    4943 generic.go:356] "Generic (PLEG): container finished" podID="762613ee-244b-4c66-baf9-903b074059bf" containerID="09cbdc112ce467bc74b36e1b9630471c9d895e5c6e9b1e6a08af0672e1458f2a" exitCode=0
Oct 30 13:06:58.886917 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:06:58.886809    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847apns8f" event={"ID":"762613ee-244b-4c66-baf9-903b074059bf","Type":"ContainerDied","Data":"09cbdc112ce467bc74b36e1b9630471c9d895e5c6e9b1e6a08af0672e1458f2a"}
Oct 30 13:06:58.886917 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:06:58.886851    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847apns8f" event={"ID":"762613ee-244b-4c66-baf9-903b074059bf","Type":"ContainerStarted","Data":"3318a6feb035248926ba064aa7757a8a8958c9e4cd0d0ced72f5f4d8416c45cb"}
Oct 30 13:06:59.892657 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:06:59.892625    4943 generic.go:356] "Generic (PLEG): container finished" podID="762613ee-244b-4c66-baf9-903b074059bf" containerID="6adcdc4030744cb2a67f58b8b540c0871db91fc6c0a18e7293a9256ec4aa4c2f" exitCode=0
Oct 30 13:06:59.893036 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:06:59.892709    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847apns8f" event={"ID":"762613ee-244b-4c66-baf9-903b074059bf","Type":"ContainerDied","Data":"6adcdc4030744cb2a67f58b8b540c0871db91fc6c0a18e7293a9256ec4aa4c2f"}
Oct 30 13:07:00.899079 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:07:00.899047    4943 generic.go:356] "Generic (PLEG): container finished" podID="762613ee-244b-4c66-baf9-903b074059bf" containerID="966dbb244603ec39f6c7a75b83c1e0b5457d4408dd240250d3d73acf8c6c0f48" exitCode=0
Oct 30 13:07:00.899501 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:07:00.899136    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847apns8f" event={"ID":"762613ee-244b-4c66-baf9-903b074059bf","Type":"ContainerDied","Data":"966dbb244603ec39f6c7a75b83c1e0b5457d4408dd240250d3d73acf8c6c0f48"}
Oct 30 13:07:02.029706 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:07:02.029683    4943 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847apns8f"
Oct 30 13:07:02.134094 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:07:02.134064    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/762613ee-244b-4c66-baf9-903b074059bf-kube-api-access-7tp46\" (UniqueName: \"kubernetes.io/projected/762613ee-244b-4c66-baf9-903b074059bf-kube-api-access-7tp46\") pod \"762613ee-244b-4c66-baf9-903b074059bf\" (UID: \"762613ee-244b-4c66-baf9-903b074059bf\") "
Oct 30 13:07:02.134236 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:07:02.134113    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/762613ee-244b-4c66-baf9-903b074059bf-util\" (UniqueName: \"kubernetes.io/empty-dir/762613ee-244b-4c66-baf9-903b074059bf-util\") pod \"762613ee-244b-4c66-baf9-903b074059bf\" (UID: \"762613ee-244b-4c66-baf9-903b074059bf\") "
Oct 30 13:07:02.134236 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:07:02.134189    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/762613ee-244b-4c66-baf9-903b074059bf-bundle\" (UniqueName: \"kubernetes.io/empty-dir/762613ee-244b-4c66-baf9-903b074059bf-bundle\") pod \"762613ee-244b-4c66-baf9-903b074059bf\" (UID: \"762613ee-244b-4c66-baf9-903b074059bf\") "
Oct 30 13:07:02.135007 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:07:02.134968    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/762613ee-244b-4c66-baf9-903b074059bf-bundle" pod "762613ee-244b-4c66-baf9-903b074059bf" (UID: "762613ee-244b-4c66-baf9-903b074059bf"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:07:02.136202 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:07:02.136174    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/762613ee-244b-4c66-baf9-903b074059bf-kube-api-access-7tp46" pod "762613ee-244b-4c66-baf9-903b074059bf" (UID: "762613ee-244b-4c66-baf9-903b074059bf"). InnerVolumeSpecName "kube-api-access-7tp46". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:07:02.139791 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:07:02.139769    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/762613ee-244b-4c66-baf9-903b074059bf-util" pod "762613ee-244b-4c66-baf9-903b074059bf" (UID: "762613ee-244b-4c66-baf9-903b074059bf"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:07:02.235413 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:07:02.235388    4943 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/762613ee-244b-4c66-baf9-903b074059bf-bundle\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:07:02.235413 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:07:02.235414    4943 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7tp46\" (UniqueName: \"kubernetes.io/projected/762613ee-244b-4c66-baf9-903b074059bf-kube-api-access-7tp46\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:07:02.235560 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:07:02.235424    4943 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/762613ee-244b-4c66-baf9-903b074059bf-util\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:07:02.911732 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:07:02.911703    4943 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847apns8f"
Oct 30 13:07:02.911732 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:07:02.911710    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847apns8f" event={"ID":"762613ee-244b-4c66-baf9-903b074059bf","Type":"ContainerDied","Data":"3318a6feb035248926ba064aa7757a8a8958c9e4cd0d0ced72f5f4d8416c45cb"}
Oct 30 13:07:02.911938 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:07:02.911751    4943 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3318a6feb035248926ba064aa7757a8a8958c9e4cd0d0ced72f5f4d8416c45cb"
Oct 30 13:07:04.389867 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:07:04.389839    4943 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ajkfrv_17f5a8d2-67e0-4489-8734-00f0d4483788/extract/0.log"
Oct 30 13:07:04.390333 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:07:04.389839    4943 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ajkfrv_17f5a8d2-67e0-4489-8734-00f0d4483788/extract/0.log"
Oct 30 13:07:04.402029 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:07:04.402009    4943 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-8fqfr_d8930ee7-0ae0-48b5-8ce3-cb8820355cef/console-operator/2.log"
Oct 30 13:07:04.402211 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:07:04.402190    4943 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-8fqfr_d8930ee7-0ae0-48b5-8ce3-cb8820355cef/console-operator/2.log"
Oct 30 13:07:05.954292 ip-10-0-142-84 kubenswrapper[4943]: E1030 13:07:05.954236    4943 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-knqz6" containerName="extractor"
Oct 30 13:07:13.148720 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:07:13.148681    4943 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-68d6dd76dc-8fv2z"]
Oct 30 13:07:13.149352 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:07:13.149154    4943 cpu_manager.go:414] "containerMap: removing container" podUID="762613ee-244b-4c66-baf9-903b074059bf" containerName="extract"
Oct 30 13:07:13.149352 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:07:13.149172    4943 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="762613ee-244b-4c66-baf9-903b074059bf" containerName="extract"
Oct 30 13:07:13.149352 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:07:13.149199    4943 cpu_manager.go:414] "containerMap: removing container" podUID="762613ee-244b-4c66-baf9-903b074059bf" containerName="util"
Oct 30 13:07:13.149352 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:07:13.149207    4943 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="762613ee-244b-4c66-baf9-903b074059bf" containerName="util"
Oct 30 13:07:13.149352 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:07:13.149216    4943 cpu_manager.go:414] "containerMap: removing container" podUID="762613ee-244b-4c66-baf9-903b074059bf" containerName="pull"
Oct 30 13:07:13.149352 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:07:13.149226    4943 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="762613ee-244b-4c66-baf9-903b074059bf" containerName="pull"
Oct 30 13:07:13.149352 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:07:13.149350    4943 memory_manager.go:365] "RemoveStaleState removing state" podUID="762613ee-244b-4c66-baf9-903b074059bf" containerName="extract"
Oct 30 13:07:13.153787 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:07:13.153767    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-68d6dd76dc-8fv2z"
Oct 30 13:07:13.172641 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:07:13.172618    4943 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Oct 30 13:07:13.176491 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:07:13.176470    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-68d6dd76dc-8fv2z"]
Oct 30 13:07:13.186826 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:07:13.186807    4943 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Oct 30 13:07:13.187179 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:07:13.187162    4943 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-rdnph\""
Oct 30 13:07:13.219212 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:07:13.219189    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2blbg\" (UniqueName: \"kubernetes.io/projected/911d5399-ef35-4341-adb3-5d3de9c7b5fb-kube-api-access-2blbg\") pod \"authorino-operator-68d6dd76dc-8fv2z\" (UID: \"911d5399-ef35-4341-adb3-5d3de9c7b5fb\") " pod="kuadrant-system/authorino-operator-68d6dd76dc-8fv2z"
Oct 30 13:07:13.320063 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:07:13.320031    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2blbg\" (UniqueName: \"kubernetes.io/projected/911d5399-ef35-4341-adb3-5d3de9c7b5fb-kube-api-access-2blbg\") pod \"authorino-operator-68d6dd76dc-8fv2z\" (UID: \"911d5399-ef35-4341-adb3-5d3de9c7b5fb\") " pod="kuadrant-system/authorino-operator-68d6dd76dc-8fv2z"
Oct 30 13:07:13.388705 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:07:13.388673    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2blbg\" (UniqueName: \"kubernetes.io/projected/911d5399-ef35-4341-adb3-5d3de9c7b5fb-kube-api-access-2blbg\") pod \"authorino-operator-68d6dd76dc-8fv2z\" (UID: \"911d5399-ef35-4341-adb3-5d3de9c7b5fb\") " pod="kuadrant-system/authorino-operator-68d6dd76dc-8fv2z"
Oct 30 13:07:13.466362 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:07:13.466327    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-68d6dd76dc-8fv2z"
Oct 30 13:07:13.797726 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:07:13.797688    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-68d6dd76dc-8fv2z"]
Oct 30 13:07:13.798609 ip-10-0-142-84 kubenswrapper[4943]: W1030 13:07:13.798584    4943 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod911d5399_ef35_4341_adb3_5d3de9c7b5fb.slice/crio-6a6863441ab28cfccf8d8e071c122b47e73c1dc0f973a190f86708826b340f2a WatchSource:0}: Error finding container 6a6863441ab28cfccf8d8e071c122b47e73c1dc0f973a190f86708826b340f2a: Status 404 returned error can't find the container with id 6a6863441ab28cfccf8d8e071c122b47e73c1dc0f973a190f86708826b340f2a
Oct 30 13:07:13.965440 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:07:13.965387    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-68d6dd76dc-8fv2z" event={"ID":"911d5399-ef35-4341-adb3-5d3de9c7b5fb","Type":"ContainerStarted","Data":"6a6863441ab28cfccf8d8e071c122b47e73c1dc0f973a190f86708826b340f2a"}
Oct 30 13:07:16.983935 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:07:16.983900    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-68d6dd76dc-8fv2z" event={"ID":"911d5399-ef35-4341-adb3-5d3de9c7b5fb","Type":"ContainerStarted","Data":"6908941ce26af80284a45ef8a3e84b85784d5a19acc941a8a7041e9c6a810536"}
Oct 30 13:07:16.984338 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:07:16.983973    4943 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-68d6dd76dc-8fv2z"
Oct 30 13:07:17.059214 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:07:17.059166    4943 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-68d6dd76dc-8fv2z" podStartSLOduration=1.777295391 podStartE2EDuration="4.05915365s" totalImagesPullingTime="2.281858259s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:07:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:07:17.056348428 +0000 UTC m=+612.844352157" watchObservedRunningTime="2026-10-30 13:07:17.05915365 +0000 UTC m=+612.847157379"
Oct 30 13:07:20.001520 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:07:20.001485    4943 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-856dd584c5-9swbn"]
Oct 30 13:07:20.004833 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:07:20.004809    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-856dd584c5-9swbn"
Oct 30 13:07:20.037303 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:07:20.037257    4943 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-7fwpk\""
Oct 30 13:07:20.054502 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:07:20.054473    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-856dd584c5-9swbn"]
Oct 30 13:07:20.087851 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:07:20.087822    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zsvkn\" (UniqueName: \"kubernetes.io/projected/073116f6-fc23-4f35-9dbd-3afc9006df7a-kube-api-access-zsvkn\") pod \"limitador-operator-controller-manager-856dd584c5-9swbn\" (UID: \"073116f6-fc23-4f35-9dbd-3afc9006df7a\") " pod="kuadrant-system/limitador-operator-controller-manager-856dd584c5-9swbn"
Oct 30 13:07:20.188608 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:07:20.188577    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zsvkn\" (UniqueName: \"kubernetes.io/projected/073116f6-fc23-4f35-9dbd-3afc9006df7a-kube-api-access-zsvkn\") pod \"limitador-operator-controller-manager-856dd584c5-9swbn\" (UID: \"073116f6-fc23-4f35-9dbd-3afc9006df7a\") " pod="kuadrant-system/limitador-operator-controller-manager-856dd584c5-9swbn"
Oct 30 13:07:20.214689 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:07:20.214665    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zsvkn\" (UniqueName: \"kubernetes.io/projected/073116f6-fc23-4f35-9dbd-3afc9006df7a-kube-api-access-zsvkn\") pod \"limitador-operator-controller-manager-856dd584c5-9swbn\" (UID: \"073116f6-fc23-4f35-9dbd-3afc9006df7a\") " pod="kuadrant-system/limitador-operator-controller-manager-856dd584c5-9swbn"
Oct 30 13:07:20.317938 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:07:20.317864    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-856dd584c5-9swbn"
Oct 30 13:07:20.661806 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:07:20.661776    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-856dd584c5-9swbn"]
Oct 30 13:07:20.663635 ip-10-0-142-84 kubenswrapper[4943]: W1030 13:07:20.663611    4943 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod073116f6_fc23_4f35_9dbd_3afc9006df7a.slice/crio-bde656aa8ce88c7955ed49a5add5790904cf300ae74c0071a89be9dd66cc256b WatchSource:0}: Error finding container bde656aa8ce88c7955ed49a5add5790904cf300ae74c0071a89be9dd66cc256b: Status 404 returned error can't find the container with id bde656aa8ce88c7955ed49a5add5790904cf300ae74c0071a89be9dd66cc256b
Oct 30 13:07:20.665992 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:07:20.665969    4943 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 13:07:21.004017 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:07:21.003979    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-856dd584c5-9swbn" event={"ID":"073116f6-fc23-4f35-9dbd-3afc9006df7a","Type":"ContainerStarted","Data":"bde656aa8ce88c7955ed49a5add5790904cf300ae74c0071a89be9dd66cc256b"}
Oct 30 13:07:23.014329 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:07:23.014283    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-856dd584c5-9swbn" event={"ID":"073116f6-fc23-4f35-9dbd-3afc9006df7a","Type":"ContainerStarted","Data":"a08fbe6f3a85222b81b92aeced814996bc5bb39266f3c80721d656b457672a4a"}
Oct 30 13:07:23.014774 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:07:23.014356    4943 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-856dd584c5-9swbn"
Oct 30 13:07:23.078015 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:07:23.077962    4943 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-856dd584c5-9swbn" podStartSLOduration=2.774488059 podStartE2EDuration="4.077947542s" totalImagesPullingTime="1.303459483s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:07:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:07:23.07622784 +0000 UTC m=+618.864231571" watchObservedRunningTime="2026-10-30 13:07:23.077947542 +0000 UTC m=+618.865951272"
Oct 30 13:07:27.991387 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:07:27.991351    4943 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-68d6dd76dc-8fv2z"
Oct 30 13:07:34.021837 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:07:34.021805    4943 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-856dd584c5-9swbn"
Oct 30 13:07:35.954471 ip-10-0-142-84 kubenswrapper[4943]: E1030 13:07:35.954433    4943 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-knqz6" containerName="extractor"
Oct 30 13:07:38.159147 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:07:38.159113    4943 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-856dd584c5-9swbn"]
Oct 30 13:07:38.159570 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:07:38.159374    4943 kuberuntime_container.go:898] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-856dd584c5-9swbn" podUID="073116f6-fc23-4f35-9dbd-3afc9006df7a" containerName="manager" containerID="cri-o://a08fbe6f3a85222b81b92aeced814996bc5bb39266f3c80721d656b457672a4a" gracePeriod=2
Oct 30 13:07:38.175566 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:07:38.175536    4943 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-856dd584c5-9swbn"]
Oct 30 13:07:38.175687 ip-10-0-142-84 kubenswrapper[4943]: E1030 13:07:38.175532    4943 status_manager.go:1045] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-856dd584c5-9swbn\" is forbidden: User \"system:node:ip-10-0-142-84.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-84.ec2.internal' and this object" podUID="073116f6-fc23-4f35-9dbd-3afc9006df7a" pod="kuadrant-system/limitador-operator-controller-manager-856dd584c5-9swbn"
Oct 30 13:07:38.188073 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:07:38.188043    4943 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-856dd584c5-hlq89"]
Oct 30 13:07:38.188404 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:07:38.188390    4943 cpu_manager.go:414] "containerMap: removing container" podUID="073116f6-fc23-4f35-9dbd-3afc9006df7a" containerName="manager"
Oct 30 13:07:38.188458 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:07:38.188406    4943 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="073116f6-fc23-4f35-9dbd-3afc9006df7a" containerName="manager"
Oct 30 13:07:38.188505 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:07:38.188494    4943 memory_manager.go:365] "RemoveStaleState removing state" podUID="073116f6-fc23-4f35-9dbd-3afc9006df7a" containerName="manager"
Oct 30 13:07:38.191361 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:07:38.191341    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-856dd584c5-hlq89"
Oct 30 13:07:38.207238 ip-10-0-142-84 kubenswrapper[4943]: E1030 13:07:38.207205    4943 status_manager.go:1045] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-856dd584c5-9swbn\" is forbidden: User \"system:node:ip-10-0-142-84.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-84.ec2.internal' and this object" podUID="073116f6-fc23-4f35-9dbd-3afc9006df7a" pod="kuadrant-system/limitador-operator-controller-manager-856dd584c5-9swbn"
Oct 30 13:07:38.215453 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:07:38.215427    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-856dd584c5-hlq89"]
Oct 30 13:07:38.344020 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:07:38.343983    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7527f\" (UniqueName: \"kubernetes.io/projected/a303f0b6-1d9d-46a1-9309-43e09bec7618-kube-api-access-7527f\") pod \"limitador-operator-controller-manager-856dd584c5-hlq89\" (UID: \"a303f0b6-1d9d-46a1-9309-43e09bec7618\") " pod="kuadrant-system/limitador-operator-controller-manager-856dd584c5-hlq89"
Oct 30 13:07:38.405601 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:07:38.405576    4943 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-856dd584c5-9swbn"
Oct 30 13:07:38.444610 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:07:38.444584    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7527f\" (UniqueName: \"kubernetes.io/projected/a303f0b6-1d9d-46a1-9309-43e09bec7618-kube-api-access-7527f\") pod \"limitador-operator-controller-manager-856dd584c5-hlq89\" (UID: \"a303f0b6-1d9d-46a1-9309-43e09bec7618\") " pod="kuadrant-system/limitador-operator-controller-manager-856dd584c5-hlq89"
Oct 30 13:07:38.475006 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:07:38.474978    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7527f\" (UniqueName: \"kubernetes.io/projected/a303f0b6-1d9d-46a1-9309-43e09bec7618-kube-api-access-7527f\") pod \"limitador-operator-controller-manager-856dd584c5-hlq89\" (UID: \"a303f0b6-1d9d-46a1-9309-43e09bec7618\") " pod="kuadrant-system/limitador-operator-controller-manager-856dd584c5-hlq89"
Oct 30 13:07:38.533908 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:07:38.533883    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-856dd584c5-hlq89"
Oct 30 13:07:38.545802 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:07:38.545777    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/073116f6-fc23-4f35-9dbd-3afc9006df7a-kube-api-access-zsvkn\" (UniqueName: \"kubernetes.io/projected/073116f6-fc23-4f35-9dbd-3afc9006df7a-kube-api-access-zsvkn\") pod \"073116f6-fc23-4f35-9dbd-3afc9006df7a\" (UID: \"073116f6-fc23-4f35-9dbd-3afc9006df7a\") "
Oct 30 13:07:38.548100 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:07:38.548068    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/073116f6-fc23-4f35-9dbd-3afc9006df7a-kube-api-access-zsvkn" pod "073116f6-fc23-4f35-9dbd-3afc9006df7a" (UID: "073116f6-fc23-4f35-9dbd-3afc9006df7a"). InnerVolumeSpecName "kube-api-access-zsvkn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:07:38.646623 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:07:38.646591    4943 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zsvkn\" (UniqueName: \"kubernetes.io/projected/073116f6-fc23-4f35-9dbd-3afc9006df7a-kube-api-access-zsvkn\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:07:38.836508 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:07:38.836482    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-856dd584c5-hlq89"]
Oct 30 13:07:38.838246 ip-10-0-142-84 kubenswrapper[4943]: W1030 13:07:38.838225    4943 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda303f0b6_1d9d_46a1_9309_43e09bec7618.slice/crio-c3da15013850b786e5a8335acf5bbdfd66d1d5484baeaa84dbbc133dbf2dd38e WatchSource:0}: Error finding container c3da15013850b786e5a8335acf5bbdfd66d1d5484baeaa84dbbc133dbf2dd38e: Status 404 returned error can't find the container with id c3da15013850b786e5a8335acf5bbdfd66d1d5484baeaa84dbbc133dbf2dd38e
Oct 30 13:07:39.099633 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:07:39.099553    4943 generic.go:356] "Generic (PLEG): container finished" podID="073116f6-fc23-4f35-9dbd-3afc9006df7a" containerID="a08fbe6f3a85222b81b92aeced814996bc5bb39266f3c80721d656b457672a4a" exitCode=0
Oct 30 13:07:39.099633 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:07:39.099597    4943 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-856dd584c5-9swbn"
Oct 30 13:07:39.099811 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:07:39.099651    4943 scope.go:122] "RemoveContainer" containerID="a08fbe6f3a85222b81b92aeced814996bc5bb39266f3c80721d656b457672a4a"
Oct 30 13:07:39.101491 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:07:39.101461    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-856dd584c5-hlq89" event={"ID":"a303f0b6-1d9d-46a1-9309-43e09bec7618","Type":"ContainerStarted","Data":"9e688d43e255f5dc17479dc856928ceac0f8318b15042712bf69b362cae48495"}
Oct 30 13:07:39.101793 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:07:39.101509    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-856dd584c5-hlq89" event={"ID":"a303f0b6-1d9d-46a1-9309-43e09bec7618","Type":"ContainerStarted","Data":"c3da15013850b786e5a8335acf5bbdfd66d1d5484baeaa84dbbc133dbf2dd38e"}
Oct 30 13:07:39.101793 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:07:39.101585    4943 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-856dd584c5-hlq89"
Oct 30 13:07:39.110041 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:07:39.110023    4943 scope.go:122] "RemoveContainer" containerID="a08fbe6f3a85222b81b92aeced814996bc5bb39266f3c80721d656b457672a4a"
Oct 30 13:07:39.110308 ip-10-0-142-84 kubenswrapper[4943]: E1030 13:07:39.110289    4943 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a08fbe6f3a85222b81b92aeced814996bc5bb39266f3c80721d656b457672a4a\": container with ID starting with a08fbe6f3a85222b81b92aeced814996bc5bb39266f3c80721d656b457672a4a not found: ID does not exist" containerID="a08fbe6f3a85222b81b92aeced814996bc5bb39266f3c80721d656b457672a4a"
Oct 30 13:07:39.110380 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:07:39.110322    4943 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a08fbe6f3a85222b81b92aeced814996bc5bb39266f3c80721d656b457672a4a"} err="failed to get container status \"a08fbe6f3a85222b81b92aeced814996bc5bb39266f3c80721d656b457672a4a\": rpc error: code = NotFound desc = could not find container \"a08fbe6f3a85222b81b92aeced814996bc5bb39266f3c80721d656b457672a4a\": container with ID starting with a08fbe6f3a85222b81b92aeced814996bc5bb39266f3c80721d656b457672a4a not found: ID does not exist"
Oct 30 13:07:39.163618 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:07:39.163562    4943 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-856dd584c5-hlq89" podStartSLOduration=1.163548263 podStartE2EDuration="1.163548263s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:07:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:07:39.162840953 +0000 UTC m=+634.950844684" watchObservedRunningTime="2026-10-30 13:07:39.163548263 +0000 UTC m=+634.951551993"
Oct 30 13:07:40.348912 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:07:40.348884    4943 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="073116f6-fc23-4f35-9dbd-3afc9006df7a" path="/var/lib/kubelet/pods/073116f6-fc23-4f35-9dbd-3afc9006df7a/volumes"
Oct 30 13:07:50.109282 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:07:50.109236    4943 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-856dd584c5-hlq89"
Oct 30 13:08:05.953880 ip-10-0-142-84 kubenswrapper[4943]: E1030 13:08:05.953837    4943 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-knqz6" containerName="extractor"
Oct 30 13:08:21.493613 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:08:21.493528    4943 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-57f7f8659f-r4446"]
Oct 30 13:08:21.496152 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:08:21.496119    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-57f7f8659f-r4446"
Oct 30 13:08:21.515050 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:08:21.515014    4943 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-m59kx\""
Oct 30 13:08:21.516184 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:08:21.516164    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-57f7f8659f-r4446"]
Oct 30 13:08:21.582943 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:08:21.582916    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vtlzv\" (UniqueName: \"kubernetes.io/projected/70cd66ea-ad94-412d-ae2f-7b44b4a32723-kube-api-access-vtlzv\") pod \"authorino-57f7f8659f-r4446\" (UID: \"70cd66ea-ad94-412d-ae2f-7b44b4a32723\") " pod="kuadrant-system/authorino-57f7f8659f-r4446"
Oct 30 13:08:21.683448 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:08:21.683423    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vtlzv\" (UniqueName: \"kubernetes.io/projected/70cd66ea-ad94-412d-ae2f-7b44b4a32723-kube-api-access-vtlzv\") pod \"authorino-57f7f8659f-r4446\" (UID: \"70cd66ea-ad94-412d-ae2f-7b44b4a32723\") " pod="kuadrant-system/authorino-57f7f8659f-r4446"
Oct 30 13:08:21.709931 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:08:21.709905    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vtlzv\" (UniqueName: \"kubernetes.io/projected/70cd66ea-ad94-412d-ae2f-7b44b4a32723-kube-api-access-vtlzv\") pod \"authorino-57f7f8659f-r4446\" (UID: \"70cd66ea-ad94-412d-ae2f-7b44b4a32723\") " pod="kuadrant-system/authorino-57f7f8659f-r4446"
Oct 30 13:08:21.807433 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:08:21.807376    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-57f7f8659f-r4446"
Oct 30 13:08:22.103278 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:08:22.103184    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-57f7f8659f-r4446"]
Oct 30 13:08:22.105749 ip-10-0-142-84 kubenswrapper[4943]: W1030 13:08:22.105722    4943 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod70cd66ea_ad94_412d_ae2f_7b44b4a32723.slice/crio-b63d187f33f9082669d02722605fb46cf72bfb8e75ae60ac761ae2e4e9b78f51 WatchSource:0}: Error finding container b63d187f33f9082669d02722605fb46cf72bfb8e75ae60ac761ae2e4e9b78f51: Status 404 returned error can't find the container with id b63d187f33f9082669d02722605fb46cf72bfb8e75ae60ac761ae2e4e9b78f51
Oct 30 13:08:22.200430 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:08:22.200390    4943 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-hdmrm"]
Oct 30 13:08:22.203474 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:08:22.203453    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-7c9b87d574-hdmrm"
Oct 30 13:08:22.224397 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:08:22.224373    4943 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Oct 30 13:08:22.226938 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:08:22.226915    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-hdmrm"]
Oct 30 13:08:22.273467 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:08:22.273437    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-hdmrm"]
Oct 30 13:08:22.287826 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:08:22.287799    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fzzwv\" (UniqueName: \"kubernetes.io/projected/306df0eb-65a9-4f3f-b55b-fb4389c208e8-kube-api-access-fzzwv\") pod \"limitador-limitador-7c9b87d574-hdmrm\" (UID: \"306df0eb-65a9-4f3f-b55b-fb4389c208e8\") " pod="kuadrant-system/limitador-limitador-7c9b87d574-hdmrm"
Oct 30 13:08:22.287936 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:08:22.287865    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/306df0eb-65a9-4f3f-b55b-fb4389c208e8-config-file\") pod \"limitador-limitador-7c9b87d574-hdmrm\" (UID: \"306df0eb-65a9-4f3f-b55b-fb4389c208e8\") " pod="kuadrant-system/limitador-limitador-7c9b87d574-hdmrm"
Oct 30 13:08:22.309408 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:08:22.309378    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-57f7f8659f-r4446" event={"ID":"70cd66ea-ad94-412d-ae2f-7b44b4a32723","Type":"ContainerStarted","Data":"b63d187f33f9082669d02722605fb46cf72bfb8e75ae60ac761ae2e4e9b78f51"}
Oct 30 13:08:22.312242 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:08:22.312225    4943 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-t5j5x\""
Oct 30 13:08:22.388309 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:08:22.388235    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/306df0eb-65a9-4f3f-b55b-fb4389c208e8-config-file\") pod \"limitador-limitador-7c9b87d574-hdmrm\" (UID: \"306df0eb-65a9-4f3f-b55b-fb4389c208e8\") " pod="kuadrant-system/limitador-limitador-7c9b87d574-hdmrm"
Oct 30 13:08:22.388309 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:08:22.388297    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fzzwv\" (UniqueName: \"kubernetes.io/projected/306df0eb-65a9-4f3f-b55b-fb4389c208e8-kube-api-access-fzzwv\") pod \"limitador-limitador-7c9b87d574-hdmrm\" (UID: \"306df0eb-65a9-4f3f-b55b-fb4389c208e8\") " pod="kuadrant-system/limitador-limitador-7c9b87d574-hdmrm"
Oct 30 13:08:22.388843 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:08:22.388818    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/306df0eb-65a9-4f3f-b55b-fb4389c208e8-config-file\") pod \"limitador-limitador-7c9b87d574-hdmrm\" (UID: \"306df0eb-65a9-4f3f-b55b-fb4389c208e8\") " pod="kuadrant-system/limitador-limitador-7c9b87d574-hdmrm"
Oct 30 13:08:22.413396 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:08:22.413373    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fzzwv\" (UniqueName: \"kubernetes.io/projected/306df0eb-65a9-4f3f-b55b-fb4389c208e8-kube-api-access-fzzwv\") pod \"limitador-limitador-7c9b87d574-hdmrm\" (UID: \"306df0eb-65a9-4f3f-b55b-fb4389c208e8\") " pod="kuadrant-system/limitador-limitador-7c9b87d574-hdmrm"
Oct 30 13:08:22.516204 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:08:22.516173    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-7c9b87d574-hdmrm"
Oct 30 13:08:22.818555 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:08:22.818527    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-hdmrm"]
Oct 30 13:08:23.317897 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:08:23.317855    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-7c9b87d574-hdmrm" event={"ID":"306df0eb-65a9-4f3f-b55b-fb4389c208e8","Type":"ContainerStarted","Data":"30fca5821df52d8c355b3748897e636f99c0056a2c6c1b66d29443c4563142ea"}
Oct 30 13:08:24.324975 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:08:24.324928    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-7c9b87d574-hdmrm" event={"ID":"306df0eb-65a9-4f3f-b55b-fb4389c208e8","Type":"ContainerStarted","Data":"566e031a1bf046527ade2f9d641c85f1f165b8ab79664c741c689830bc991846"}
Oct 30 13:08:24.325472 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:08:24.325034    4943 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-7c9b87d574-hdmrm"
Oct 30 13:08:24.326432 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:08:24.326408    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-57f7f8659f-r4446" event={"ID":"70cd66ea-ad94-412d-ae2f-7b44b4a32723","Type":"ContainerStarted","Data":"99e40b33343991e7842d5b575a2e0ef9905b2fc6099d7078886db266a2c85428"}
Oct 30 13:08:24.370787 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:08:24.370732    4943 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-7c9b87d574-hdmrm" podStartSLOduration=1.5477339730000002 podStartE2EDuration="2.370715095s" totalImagesPullingTime="822.981122ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:08:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:08:24.369294799 +0000 UTC m=+680.157298530" watchObservedRunningTime="2026-10-30 13:08:24.370715095 +0000 UTC m=+680.158718827"
Oct 30 13:08:24.415777 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:08:24.415731    4943 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/authorino-57f7f8659f-r4446" podStartSLOduration=1.724349647 podStartE2EDuration="3.415717878s" totalImagesPullingTime="1.691368231s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:08:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:08:24.414309561 +0000 UTC m=+680.202313295" watchObservedRunningTime="2026-10-30 13:08:24.415717878 +0000 UTC m=+680.203721613"
Oct 30 13:08:24.662225 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:08:24.662150    4943 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-57f7f8659f-r4446"]
Oct 30 13:08:26.335766 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:08:26.335723    4943 kuberuntime_container.go:898] "Killing container with a grace period" pod="kuadrant-system/authorino-57f7f8659f-r4446" podUID="70cd66ea-ad94-412d-ae2f-7b44b4a32723" containerName="authorino" containerID="cri-o://99e40b33343991e7842d5b575a2e0ef9905b2fc6099d7078886db266a2c85428" gracePeriod=30
Oct 30 13:08:26.581698 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:08:26.581673    4943 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-57f7f8659f-r4446"
Oct 30 13:08:26.724051 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:08:26.724020    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/70cd66ea-ad94-412d-ae2f-7b44b4a32723-kube-api-access-vtlzv\" (UniqueName: \"kubernetes.io/projected/70cd66ea-ad94-412d-ae2f-7b44b4a32723-kube-api-access-vtlzv\") pod \"70cd66ea-ad94-412d-ae2f-7b44b4a32723\" (UID: \"70cd66ea-ad94-412d-ae2f-7b44b4a32723\") "
Oct 30 13:08:26.726073 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:08:26.726045    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/70cd66ea-ad94-412d-ae2f-7b44b4a32723-kube-api-access-vtlzv" pod "70cd66ea-ad94-412d-ae2f-7b44b4a32723" (UID: "70cd66ea-ad94-412d-ae2f-7b44b4a32723"). InnerVolumeSpecName "kube-api-access-vtlzv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:08:26.824613 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:08:26.824583    4943 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vtlzv\" (UniqueName: \"kubernetes.io/projected/70cd66ea-ad94-412d-ae2f-7b44b4a32723-kube-api-access-vtlzv\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:08:27.342688 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:08:27.342649    4943 generic.go:356] "Generic (PLEG): container finished" podID="70cd66ea-ad94-412d-ae2f-7b44b4a32723" containerID="99e40b33343991e7842d5b575a2e0ef9905b2fc6099d7078886db266a2c85428" exitCode=0
Oct 30 13:08:27.343098 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:08:27.342761    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-57f7f8659f-r4446" event={"ID":"70cd66ea-ad94-412d-ae2f-7b44b4a32723","Type":"ContainerDied","Data":"99e40b33343991e7842d5b575a2e0ef9905b2fc6099d7078886db266a2c85428"}
Oct 30 13:08:27.343098 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:08:27.342801    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-57f7f8659f-r4446" event={"ID":"70cd66ea-ad94-412d-ae2f-7b44b4a32723","Type":"ContainerDied","Data":"b63d187f33f9082669d02722605fb46cf72bfb8e75ae60ac761ae2e4e9b78f51"}
Oct 30 13:08:27.343098 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:08:27.342802    4943 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-57f7f8659f-r4446"
Oct 30 13:08:27.343098 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:08:27.342839    4943 scope.go:122] "RemoveContainer" containerID="99e40b33343991e7842d5b575a2e0ef9905b2fc6099d7078886db266a2c85428"
Oct 30 13:08:27.354897 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:08:27.354871    4943 scope.go:122] "RemoveContainer" containerID="99e40b33343991e7842d5b575a2e0ef9905b2fc6099d7078886db266a2c85428"
Oct 30 13:08:27.355195 ip-10-0-142-84 kubenswrapper[4943]: E1030 13:08:27.355164    4943 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"99e40b33343991e7842d5b575a2e0ef9905b2fc6099d7078886db266a2c85428\": container with ID starting with 99e40b33343991e7842d5b575a2e0ef9905b2fc6099d7078886db266a2c85428 not found: ID does not exist" containerID="99e40b33343991e7842d5b575a2e0ef9905b2fc6099d7078886db266a2c85428"
Oct 30 13:08:27.355242 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:08:27.355203    4943 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"99e40b33343991e7842d5b575a2e0ef9905b2fc6099d7078886db266a2c85428"} err="failed to get container status \"99e40b33343991e7842d5b575a2e0ef9905b2fc6099d7078886db266a2c85428\": rpc error: code = NotFound desc = could not find container \"99e40b33343991e7842d5b575a2e0ef9905b2fc6099d7078886db266a2c85428\": container with ID starting with 99e40b33343991e7842d5b575a2e0ef9905b2fc6099d7078886db266a2c85428 not found: ID does not exist"
Oct 30 13:08:27.400385 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:08:27.400363    4943 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-57f7f8659f-r4446"]
Oct 30 13:08:27.414056 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:08:27.414034    4943 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-57f7f8659f-r4446"]
Oct 30 13:08:28.348487 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:08:28.348453    4943 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="70cd66ea-ad94-412d-ae2f-7b44b4a32723" path="/var/lib/kubelet/pods/70cd66ea-ad94-412d-ae2f-7b44b4a32723/volumes"
Oct 30 13:08:35.332689 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:08:35.332657    4943 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-7c9b87d574-hdmrm"
Oct 30 13:08:35.953773 ip-10-0-142-84 kubenswrapper[4943]: E1030 13:08:35.953732    4943 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-knqz6" containerName="extractor"
Oct 30 13:09:05.879244 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:09:05.879205    4943 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-65d74d5fff-gstt2"]
Oct 30 13:09:05.879663 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:09:05.879595    4943 cpu_manager.go:414] "containerMap: removing container" podUID="70cd66ea-ad94-412d-ae2f-7b44b4a32723" containerName="authorino"
Oct 30 13:09:05.879663 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:09:05.879608    4943 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="70cd66ea-ad94-412d-ae2f-7b44b4a32723" containerName="authorino"
Oct 30 13:09:05.879740 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:09:05.879677    4943 memory_manager.go:365] "RemoveStaleState removing state" podUID="70cd66ea-ad94-412d-ae2f-7b44b4a32723" containerName="authorino"
Oct 30 13:09:05.884872 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:09:05.884850    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-65d74d5fff-gstt2"
Oct 30 13:09:05.902409 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:09:05.902382    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-65d74d5fff-gstt2"]
Oct 30 13:09:05.914445 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:09:05.914426    4943 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Oct 30 13:09:05.914445 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:09:05.914436    4943 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-m59kx\""
Oct 30 13:09:05.938834 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:09:05.938808    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pstmt\" (UniqueName: \"kubernetes.io/projected/be5a1e53-cca7-4305-a1dc-6882e6cd5b60-kube-api-access-pstmt\") pod \"authorino-65d74d5fff-gstt2\" (UID: \"be5a1e53-cca7-4305-a1dc-6882e6cd5b60\") " pod="kuadrant-system/authorino-65d74d5fff-gstt2"
Oct 30 13:09:05.938982 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:09:05.938861    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/be5a1e53-cca7-4305-a1dc-6882e6cd5b60-tls-cert\") pod \"authorino-65d74d5fff-gstt2\" (UID: \"be5a1e53-cca7-4305-a1dc-6882e6cd5b60\") " pod="kuadrant-system/authorino-65d74d5fff-gstt2"
Oct 30 13:09:05.954742 ip-10-0-142-84 kubenswrapper[4943]: E1030 13:09:05.954712    4943 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-knqz6" containerName="extractor"
Oct 30 13:09:06.039448 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:09:06.039422    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pstmt\" (UniqueName: \"kubernetes.io/projected/be5a1e53-cca7-4305-a1dc-6882e6cd5b60-kube-api-access-pstmt\") pod \"authorino-65d74d5fff-gstt2\" (UID: \"be5a1e53-cca7-4305-a1dc-6882e6cd5b60\") " pod="kuadrant-system/authorino-65d74d5fff-gstt2"
Oct 30 13:09:06.039553 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:09:06.039472    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/be5a1e53-cca7-4305-a1dc-6882e6cd5b60-tls-cert\") pod \"authorino-65d74d5fff-gstt2\" (UID: \"be5a1e53-cca7-4305-a1dc-6882e6cd5b60\") " pod="kuadrant-system/authorino-65d74d5fff-gstt2"
Oct 30 13:09:06.041769 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:09:06.041745    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/be5a1e53-cca7-4305-a1dc-6882e6cd5b60-tls-cert\") pod \"authorino-65d74d5fff-gstt2\" (UID: \"be5a1e53-cca7-4305-a1dc-6882e6cd5b60\") " pod="kuadrant-system/authorino-65d74d5fff-gstt2"
Oct 30 13:09:06.069118 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:09:06.069098    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pstmt\" (UniqueName: \"kubernetes.io/projected/be5a1e53-cca7-4305-a1dc-6882e6cd5b60-kube-api-access-pstmt\") pod \"authorino-65d74d5fff-gstt2\" (UID: \"be5a1e53-cca7-4305-a1dc-6882e6cd5b60\") " pod="kuadrant-system/authorino-65d74d5fff-gstt2"
Oct 30 13:09:06.196701 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:09:06.196681    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-65d74d5fff-gstt2"
Oct 30 13:09:06.519713 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:09:06.519684    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-65d74d5fff-gstt2"]
Oct 30 13:09:06.522088 ip-10-0-142-84 kubenswrapper[4943]: W1030 13:09:06.522057    4943 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbe5a1e53_cca7_4305_a1dc_6882e6cd5b60.slice/crio-152d92d7a41eb2904336e66c609b252cae6862a183da9525700581ed89452153 WatchSource:0}: Error finding container 152d92d7a41eb2904336e66c609b252cae6862a183da9525700581ed89452153: Status 404 returned error can't find the container with id 152d92d7a41eb2904336e66c609b252cae6862a183da9525700581ed89452153
Oct 30 13:09:06.534076 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:09:06.534043    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-65d74d5fff-gstt2" event={"ID":"be5a1e53-cca7-4305-a1dc-6882e6cd5b60","Type":"ContainerStarted","Data":"152d92d7a41eb2904336e66c609b252cae6862a183da9525700581ed89452153"}
Oct 30 13:09:07.540724 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:09:07.540686    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-65d74d5fff-gstt2" event={"ID":"be5a1e53-cca7-4305-a1dc-6882e6cd5b60","Type":"ContainerStarted","Data":"fb8667c69c4d108704b3d09ecdc9bb95f87ecd65f10f69b9bdb6944795f2418f"}
Oct 30 13:09:07.590601 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:09:07.590548    4943 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/authorino-65d74d5fff-gstt2" podStartSLOduration=2.092890895 podStartE2EDuration="2.590535645s" totalImagesPullingTime="497.64475ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:09:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:09:07.589695418 +0000 UTC m=+723.377699147" watchObservedRunningTime="2026-10-30 13:09:07.590535645 +0000 UTC m=+723.378539375"
Oct 30 13:09:35.954181 ip-10-0-142-84 kubenswrapper[4943]: E1030 13:09:35.954135    4943 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-knqz6" containerName="extractor"
Oct 30 13:09:53.910825 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:09:53.910739    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-hdmrm"]
Oct 30 13:10:05.954143 ip-10-0-142-84 kubenswrapper[4943]: E1030 13:10:05.954106    4943 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-knqz6" containerName="extractor"
Oct 30 13:10:35.954187 ip-10-0-142-84 kubenswrapper[4943]: E1030 13:10:35.954145    4943 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-knqz6" containerName="extractor"
Oct 30 13:10:42.876479 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:10:42.876433    4943 prober.go:122] "Probe failed" probeType="Liveness" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-84.ec2.internal" podUID="0c9cf3ec0e3ab50517e6f80b07fb5fb2" containerName="haproxy" probeResult="failure" output="Get \"https://172.20.0.1:6443/version\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:10:56.591972 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:10:56.591929    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-hdmrm"]
Oct 30 13:11:01.226579 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:01.226544    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-hdmrm"]
Oct 30 13:11:04.954164 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:04.954112    4943 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-69b588897d-jt9hf"]
Oct 30 13:11:04.958177 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:04.958149    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-jt9hf"
Oct 30 13:11:04.976382 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:04.976351    4943 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-nwjwf\""
Oct 30 13:11:04.990190 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:04.990160    4943 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\""
Oct 30 13:11:04.990190 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:04.990184    4943 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Oct 30 13:11:04.990431 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:04.990167    4943 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Oct 30 13:11:04.992404 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:04.992385    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-69b588897d-jt9hf"]
Oct 30 13:11:04.998021 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:04.997987    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6wrf6\" (UniqueName: \"kubernetes.io/projected/1d868c8e-735b-4008-a48f-d8311c5d31c5-kube-api-access-6wrf6\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-jt9hf\" (UID: \"1d868c8e-735b-4008-a48f-d8311c5d31c5\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-jt9hf"
Oct 30 13:11:04.998129 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:04.998041    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1d868c8e-735b-4008-a48f-d8311c5d31c5-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-jt9hf\" (UID: \"1d868c8e-735b-4008-a48f-d8311c5d31c5\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-jt9hf"
Oct 30 13:11:04.998129 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:04.998085    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1d868c8e-735b-4008-a48f-d8311c5d31c5-home\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-jt9hf\" (UID: \"1d868c8e-735b-4008-a48f-d8311c5d31c5\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-jt9hf"
Oct 30 13:11:04.998223 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:04.998201    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1d868c8e-735b-4008-a48f-d8311c5d31c5-model-cache\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-jt9hf\" (UID: \"1d868c8e-735b-4008-a48f-d8311c5d31c5\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-jt9hf"
Oct 30 13:11:04.998298 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:04.998235    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1d868c8e-735b-4008-a48f-d8311c5d31c5-dshm\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-jt9hf\" (UID: \"1d868c8e-735b-4008-a48f-d8311c5d31c5\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-jt9hf"
Oct 30 13:11:04.998363 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:04.998309    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1d868c8e-735b-4008-a48f-d8311c5d31c5-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-jt9hf\" (UID: \"1d868c8e-735b-4008-a48f-d8311c5d31c5\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-jt9hf"
Oct 30 13:11:05.099309 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:05.099277    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1d868c8e-735b-4008-a48f-d8311c5d31c5-dshm\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-jt9hf\" (UID: \"1d868c8e-735b-4008-a48f-d8311c5d31c5\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-jt9hf"
Oct 30 13:11:05.099309 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:05.099308    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1d868c8e-735b-4008-a48f-d8311c5d31c5-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-jt9hf\" (UID: \"1d868c8e-735b-4008-a48f-d8311c5d31c5\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-jt9hf"
Oct 30 13:11:05.099534 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:05.099357    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6wrf6\" (UniqueName: \"kubernetes.io/projected/1d868c8e-735b-4008-a48f-d8311c5d31c5-kube-api-access-6wrf6\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-jt9hf\" (UID: \"1d868c8e-735b-4008-a48f-d8311c5d31c5\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-jt9hf"
Oct 30 13:11:05.099534 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:05.099380    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1d868c8e-735b-4008-a48f-d8311c5d31c5-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-jt9hf\" (UID: \"1d868c8e-735b-4008-a48f-d8311c5d31c5\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-jt9hf"
Oct 30 13:11:05.099534 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:05.099406    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1d868c8e-735b-4008-a48f-d8311c5d31c5-home\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-jt9hf\" (UID: \"1d868c8e-735b-4008-a48f-d8311c5d31c5\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-jt9hf"
Oct 30 13:11:05.099534 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:05.099454    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1d868c8e-735b-4008-a48f-d8311c5d31c5-model-cache\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-jt9hf\" (UID: \"1d868c8e-735b-4008-a48f-d8311c5d31c5\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-jt9hf"
Oct 30 13:11:05.099823 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:05.099802    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1d868c8e-735b-4008-a48f-d8311c5d31c5-model-cache\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-jt9hf\" (UID: \"1d868c8e-735b-4008-a48f-d8311c5d31c5\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-jt9hf"
Oct 30 13:11:05.099885 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:05.099862    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1d868c8e-735b-4008-a48f-d8311c5d31c5-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-jt9hf\" (UID: \"1d868c8e-735b-4008-a48f-d8311c5d31c5\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-jt9hf"
Oct 30 13:11:05.099920 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:05.099892    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1d868c8e-735b-4008-a48f-d8311c5d31c5-home\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-jt9hf\" (UID: \"1d868c8e-735b-4008-a48f-d8311c5d31c5\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-jt9hf"
Oct 30 13:11:05.101562 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:05.101536    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1d868c8e-735b-4008-a48f-d8311c5d31c5-dshm\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-jt9hf\" (UID: \"1d868c8e-735b-4008-a48f-d8311c5d31c5\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-jt9hf"
Oct 30 13:11:05.101854 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:05.101837    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1d868c8e-735b-4008-a48f-d8311c5d31c5-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-jt9hf\" (UID: \"1d868c8e-735b-4008-a48f-d8311c5d31c5\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-jt9hf"
Oct 30 13:11:05.129216 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:05.129193    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6wrf6\" (UniqueName: \"kubernetes.io/projected/1d868c8e-735b-4008-a48f-d8311c5d31c5-kube-api-access-6wrf6\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-jt9hf\" (UID: \"1d868c8e-735b-4008-a48f-d8311c5d31c5\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-jt9hf"
Oct 30 13:11:05.274731 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:05.274651    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-jt9hf"
Oct 30 13:11:05.582217 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:05.582185    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-69b588897d-jt9hf"]
Oct 30 13:11:05.584353 ip-10-0-142-84 kubenswrapper[4943]: W1030 13:11:05.584328    4943 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1d868c8e_735b_4008_a48f_d8311c5d31c5.slice/crio-8bf465713a94505e2e3ce868af3487ab1b517df41865b2deb3a2ab42d929ffef WatchSource:0}: Error finding container 8bf465713a94505e2e3ce868af3487ab1b517df41865b2deb3a2ab42d929ffef: Status 404 returned error can't find the container with id 8bf465713a94505e2e3ce868af3487ab1b517df41865b2deb3a2ab42d929ffef
Oct 30 13:11:05.965321 ip-10-0-142-84 kubenswrapper[4943]: E1030 13:11:05.965256    4943 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-knqz6" containerName="extractor"
Oct 30 13:11:06.147979 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:06.147939    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-jt9hf" event={"ID":"1d868c8e-735b-4008-a48f-d8311c5d31c5","Type":"ContainerStarted","Data":"8bf465713a94505e2e3ce868af3487ab1b517df41865b2deb3a2ab42d929ffef"}
Oct 30 13:11:06.693055 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:06.693028    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-hdmrm"]
Oct 30 13:11:07.155775 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:07.155733    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-jt9hf" event={"ID":"1d868c8e-735b-4008-a48f-d8311c5d31c5","Type":"ContainerStarted","Data":"dfbbb5c20acf8f7fd048a452b30993c801994d78a96e334621973e0edb91ba5c"}
Oct 30 13:11:07.202309 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:07.202232    4943 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-jt9hf" podStartSLOduration=2.145402328 podStartE2EDuration="3.202212214s" totalImagesPullingTime="1.056809886s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:11:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:11:07.202184661 +0000 UTC m=+842.990188392" watchObservedRunningTime="2026-10-30 13:11:07.202212214 +0000 UTC m=+842.990215944"
Oct 30 13:11:07.274803 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:07.274762    4943 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-jt9hf"
Oct 30 13:11:07.895193 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:07.895154    4943 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8kv25h"]
Oct 30 13:11:07.906736 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:07.906689    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8kv25h"
Oct 30 13:11:07.921620 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:07.921592    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8kv25h"]
Oct 30 13:11:07.926629 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:07.926604    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ghh72\" (UniqueName: \"kubernetes.io/projected/de3d7750-cd0c-4fbf-9ccf-b061d5d20e04-kube-api-access-ghh72\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8kv25h\" (UID: \"de3d7750-cd0c-4fbf-9ccf-b061d5d20e04\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8kv25h"
Oct 30 13:11:07.926731 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:07.926650    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/de3d7750-cd0c-4fbf-9ccf-b061d5d20e04-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8kv25h\" (UID: \"de3d7750-cd0c-4fbf-9ccf-b061d5d20e04\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8kv25h"
Oct 30 13:11:07.926777 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:07.926742    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/de3d7750-cd0c-4fbf-9ccf-b061d5d20e04-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8kv25h\" (UID: \"de3d7750-cd0c-4fbf-9ccf-b061d5d20e04\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8kv25h"
Oct 30 13:11:07.926777 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:07.926763    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/de3d7750-cd0c-4fbf-9ccf-b061d5d20e04-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8kv25h\" (UID: \"de3d7750-cd0c-4fbf-9ccf-b061d5d20e04\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8kv25h"
Oct 30 13:11:07.926844 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:07.926789    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/de3d7750-cd0c-4fbf-9ccf-b061d5d20e04-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8kv25h\" (UID: \"de3d7750-cd0c-4fbf-9ccf-b061d5d20e04\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8kv25h"
Oct 30 13:11:07.926844 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:07.926834    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/de3d7750-cd0c-4fbf-9ccf-b061d5d20e04-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8kv25h\" (UID: \"de3d7750-cd0c-4fbf-9ccf-b061d5d20e04\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8kv25h"
Oct 30 13:11:07.933468 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:07.933447    4943 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\""
Oct 30 13:11:08.027732 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:08.027701    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/de3d7750-cd0c-4fbf-9ccf-b061d5d20e04-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8kv25h\" (UID: \"de3d7750-cd0c-4fbf-9ccf-b061d5d20e04\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8kv25h"
Oct 30 13:11:08.027890 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:08.027746    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ghh72\" (UniqueName: \"kubernetes.io/projected/de3d7750-cd0c-4fbf-9ccf-b061d5d20e04-kube-api-access-ghh72\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8kv25h\" (UID: \"de3d7750-cd0c-4fbf-9ccf-b061d5d20e04\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8kv25h"
Oct 30 13:11:08.027890 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:08.027781    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/de3d7750-cd0c-4fbf-9ccf-b061d5d20e04-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8kv25h\" (UID: \"de3d7750-cd0c-4fbf-9ccf-b061d5d20e04\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8kv25h"
Oct 30 13:11:08.027890 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:08.027830    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/de3d7750-cd0c-4fbf-9ccf-b061d5d20e04-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8kv25h\" (UID: \"de3d7750-cd0c-4fbf-9ccf-b061d5d20e04\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8kv25h"
Oct 30 13:11:08.027890 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:08.027850    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/de3d7750-cd0c-4fbf-9ccf-b061d5d20e04-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8kv25h\" (UID: \"de3d7750-cd0c-4fbf-9ccf-b061d5d20e04\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8kv25h"
Oct 30 13:11:08.028114 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:08.027890    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/de3d7750-cd0c-4fbf-9ccf-b061d5d20e04-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8kv25h\" (UID: \"de3d7750-cd0c-4fbf-9ccf-b061d5d20e04\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8kv25h"
Oct 30 13:11:08.028218 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:08.028194    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/de3d7750-cd0c-4fbf-9ccf-b061d5d20e04-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8kv25h\" (UID: \"de3d7750-cd0c-4fbf-9ccf-b061d5d20e04\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8kv25h"
Oct 30 13:11:08.028311 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:08.028216    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/de3d7750-cd0c-4fbf-9ccf-b061d5d20e04-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8kv25h\" (UID: \"de3d7750-cd0c-4fbf-9ccf-b061d5d20e04\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8kv25h"
Oct 30 13:11:08.028311 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:08.028248    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/de3d7750-cd0c-4fbf-9ccf-b061d5d20e04-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8kv25h\" (UID: \"de3d7750-cd0c-4fbf-9ccf-b061d5d20e04\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8kv25h"
Oct 30 13:11:08.030215 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:08.030190    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/de3d7750-cd0c-4fbf-9ccf-b061d5d20e04-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8kv25h\" (UID: \"de3d7750-cd0c-4fbf-9ccf-b061d5d20e04\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8kv25h"
Oct 30 13:11:08.030473 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:08.030451    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/de3d7750-cd0c-4fbf-9ccf-b061d5d20e04-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8kv25h\" (UID: \"de3d7750-cd0c-4fbf-9ccf-b061d5d20e04\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8kv25h"
Oct 30 13:11:08.080904 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:08.080879    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ghh72\" (UniqueName: \"kubernetes.io/projected/de3d7750-cd0c-4fbf-9ccf-b061d5d20e04-kube-api-access-ghh72\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8kv25h\" (UID: \"de3d7750-cd0c-4fbf-9ccf-b061d5d20e04\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8kv25h"
Oct 30 13:11:08.221890 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:08.221857    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8kv25h"
Oct 30 13:11:08.522418 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:08.522383    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8kv25h"]
Oct 30 13:11:08.524029 ip-10-0-142-84 kubenswrapper[4943]: W1030 13:11:08.523993    4943 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podde3d7750_cd0c_4fbf_9ccf_b061d5d20e04.slice/crio-4ae0085e68b3e1fe1b2747a0387126e0aec31a54674bcdf648d8689b76ed60c0 WatchSource:0}: Error finding container 4ae0085e68b3e1fe1b2747a0387126e0aec31a54674bcdf648d8689b76ed60c0: Status 404 returned error can't find the container with id 4ae0085e68b3e1fe1b2747a0387126e0aec31a54674bcdf648d8689b76ed60c0
Oct 30 13:11:09.169495 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:09.169404    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8kv25h" event={"ID":"de3d7750-cd0c-4fbf-9ccf-b061d5d20e04","Type":"ContainerStarted","Data":"77c26a85a37903860f57f01afe46632866fd333838c8b2fc674760748690c4f7"}
Oct 30 13:11:09.169495 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:09.169461    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8kv25h" event={"ID":"de3d7750-cd0c-4fbf-9ccf-b061d5d20e04","Type":"ContainerStarted","Data":"4ae0085e68b3e1fe1b2747a0387126e0aec31a54674bcdf648d8689b76ed60c0"}
Oct 30 13:11:09.216997 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:09.216949    4943 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8kv25h" podStartSLOduration=1.99432176 podStartE2EDuration="2.216936379s" totalImagesPullingTime="222.614619ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:11:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:11:09.21580392 +0000 UTC m=+845.003807672" watchObservedRunningTime="2026-10-30 13:11:09.216936379 +0000 UTC m=+845.004940109"
Oct 30 13:11:09.222816 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:09.222788    4943 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8kv25h"
Oct 30 13:11:12.387121 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:12.387081    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-hdmrm"]
Oct 30 13:11:15.274926 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:15.274895    4943 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-jt9hf"
Oct 30 13:11:15.291678 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:15.291652    4943 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-jt9hf"
Oct 30 13:11:16.214185 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:16.214156    4943 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-jt9hf"
Oct 30 13:11:18.222828 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:18.222787    4943 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8kv25h"
Oct 30 13:11:18.235106 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:18.235081    4943 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8kv25h"
Oct 30 13:11:18.302546 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:18.302512    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-hdmrm"]
Oct 30 13:11:18.719339 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:18.719307    4943 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-xs2vr"]
Oct 30 13:11:18.722749 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:18.722726    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-xs2vr"
Oct 30 13:11:18.747649 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:18.747624    4943 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-5td7d\""
Oct 30 13:11:18.763005 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:18.762981    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-xs2vr"]
Oct 30 13:11:18.833442 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:18.833412    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e3af0d52-346e-48d7-ac7d-8f2aeee62448-istio-envoy\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-xs2vr\" (UID: \"e3af0d52-346e-48d7-ac7d-8f2aeee62448\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-xs2vr"
Oct 30 13:11:18.833598 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:18.833448    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e3af0d52-346e-48d7-ac7d-8f2aeee62448-istio-token\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-xs2vr\" (UID: \"e3af0d52-346e-48d7-ac7d-8f2aeee62448\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-xs2vr"
Oct 30 13:11:18.833598 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:18.833466    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e3af0d52-346e-48d7-ac7d-8f2aeee62448-istio-data\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-xs2vr\" (UID: \"e3af0d52-346e-48d7-ac7d-8f2aeee62448\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-xs2vr"
Oct 30 13:11:18.833718 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:18.833610    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-55tg9\" (UniqueName: \"kubernetes.io/projected/e3af0d52-346e-48d7-ac7d-8f2aeee62448-kube-api-access-55tg9\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-xs2vr\" (UID: \"e3af0d52-346e-48d7-ac7d-8f2aeee62448\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-xs2vr"
Oct 30 13:11:18.833718 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:18.833660    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e3af0d52-346e-48d7-ac7d-8f2aeee62448-credential-socket\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-xs2vr\" (UID: \"e3af0d52-346e-48d7-ac7d-8f2aeee62448\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-xs2vr"
Oct 30 13:11:18.833786 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:18.833746    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e3af0d52-346e-48d7-ac7d-8f2aeee62448-workload-socket\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-xs2vr\" (UID: \"e3af0d52-346e-48d7-ac7d-8f2aeee62448\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-xs2vr"
Oct 30 13:11:18.833786 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:18.833773    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e3af0d52-346e-48d7-ac7d-8f2aeee62448-workload-certs\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-xs2vr\" (UID: \"e3af0d52-346e-48d7-ac7d-8f2aeee62448\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-xs2vr"
Oct 30 13:11:18.833893 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:18.833804    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e3af0d52-346e-48d7-ac7d-8f2aeee62448-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-xs2vr\" (UID: \"e3af0d52-346e-48d7-ac7d-8f2aeee62448\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-xs2vr"
Oct 30 13:11:18.833927 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:18.833890    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e3af0d52-346e-48d7-ac7d-8f2aeee62448-istio-podinfo\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-xs2vr\" (UID: \"e3af0d52-346e-48d7-ac7d-8f2aeee62448\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-xs2vr"
Oct 30 13:11:18.934403 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:18.934373    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e3af0d52-346e-48d7-ac7d-8f2aeee62448-istio-podinfo\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-xs2vr\" (UID: \"e3af0d52-346e-48d7-ac7d-8f2aeee62448\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-xs2vr"
Oct 30 13:11:18.934532 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:18.934417    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e3af0d52-346e-48d7-ac7d-8f2aeee62448-istio-envoy\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-xs2vr\" (UID: \"e3af0d52-346e-48d7-ac7d-8f2aeee62448\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-xs2vr"
Oct 30 13:11:18.934532 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:18.934453    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e3af0d52-346e-48d7-ac7d-8f2aeee62448-istio-token\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-xs2vr\" (UID: \"e3af0d52-346e-48d7-ac7d-8f2aeee62448\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-xs2vr"
Oct 30 13:11:18.934532 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:18.934478    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e3af0d52-346e-48d7-ac7d-8f2aeee62448-istio-data\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-xs2vr\" (UID: \"e3af0d52-346e-48d7-ac7d-8f2aeee62448\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-xs2vr"
Oct 30 13:11:18.934532 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:18.934527    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-55tg9\" (UniqueName: \"kubernetes.io/projected/e3af0d52-346e-48d7-ac7d-8f2aeee62448-kube-api-access-55tg9\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-xs2vr\" (UID: \"e3af0d52-346e-48d7-ac7d-8f2aeee62448\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-xs2vr"
Oct 30 13:11:18.934745 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:18.934561    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e3af0d52-346e-48d7-ac7d-8f2aeee62448-credential-socket\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-xs2vr\" (UID: \"e3af0d52-346e-48d7-ac7d-8f2aeee62448\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-xs2vr"
Oct 30 13:11:18.934745 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:18.934594    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e3af0d52-346e-48d7-ac7d-8f2aeee62448-workload-socket\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-xs2vr\" (UID: \"e3af0d52-346e-48d7-ac7d-8f2aeee62448\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-xs2vr"
Oct 30 13:11:18.934745 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:18.934618    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e3af0d52-346e-48d7-ac7d-8f2aeee62448-workload-certs\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-xs2vr\" (UID: \"e3af0d52-346e-48d7-ac7d-8f2aeee62448\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-xs2vr"
Oct 30 13:11:18.934745 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:18.934654    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e3af0d52-346e-48d7-ac7d-8f2aeee62448-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-xs2vr\" (UID: \"e3af0d52-346e-48d7-ac7d-8f2aeee62448\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-xs2vr"
Oct 30 13:11:18.934996 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:18.934971    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e3af0d52-346e-48d7-ac7d-8f2aeee62448-istio-data\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-xs2vr\" (UID: \"e3af0d52-346e-48d7-ac7d-8f2aeee62448\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-xs2vr"
Oct 30 13:11:18.935088 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:18.934994    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e3af0d52-346e-48d7-ac7d-8f2aeee62448-credential-socket\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-xs2vr\" (UID: \"e3af0d52-346e-48d7-ac7d-8f2aeee62448\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-xs2vr"
Oct 30 13:11:18.935147 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:18.935114    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e3af0d52-346e-48d7-ac7d-8f2aeee62448-workload-socket\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-xs2vr\" (UID: \"e3af0d52-346e-48d7-ac7d-8f2aeee62448\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-xs2vr"
Oct 30 13:11:18.935208 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:18.935167    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e3af0d52-346e-48d7-ac7d-8f2aeee62448-workload-certs\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-xs2vr\" (UID: \"e3af0d52-346e-48d7-ac7d-8f2aeee62448\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-xs2vr"
Oct 30 13:11:18.935462 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:18.935442    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e3af0d52-346e-48d7-ac7d-8f2aeee62448-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-xs2vr\" (UID: \"e3af0d52-346e-48d7-ac7d-8f2aeee62448\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-xs2vr"
Oct 30 13:11:18.937293 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:18.937270    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e3af0d52-346e-48d7-ac7d-8f2aeee62448-istio-envoy\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-xs2vr\" (UID: \"e3af0d52-346e-48d7-ac7d-8f2aeee62448\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-xs2vr"
Oct 30 13:11:18.937472 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:18.937454    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e3af0d52-346e-48d7-ac7d-8f2aeee62448-istio-podinfo\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-xs2vr\" (UID: \"e3af0d52-346e-48d7-ac7d-8f2aeee62448\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-xs2vr"
Oct 30 13:11:18.961096 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:18.961071    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e3af0d52-346e-48d7-ac7d-8f2aeee62448-istio-token\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-xs2vr\" (UID: \"e3af0d52-346e-48d7-ac7d-8f2aeee62448\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-xs2vr"
Oct 30 13:11:18.961446 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:18.961429    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-55tg9\" (UniqueName: \"kubernetes.io/projected/e3af0d52-346e-48d7-ac7d-8f2aeee62448-kube-api-access-55tg9\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-xs2vr\" (UID: \"e3af0d52-346e-48d7-ac7d-8f2aeee62448\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-xs2vr"
Oct 30 13:11:19.035979 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:19.035927    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-xs2vr"
Oct 30 13:11:19.229227 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:19.229192    4943 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8kv25h"
Oct 30 13:11:19.351944 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:19.351921    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-xs2vr"]
Oct 30 13:11:19.353789 ip-10-0-142-84 kubenswrapper[4943]: W1030 13:11:19.353762    4943 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode3af0d52_346e_48d7_ac7d_8f2aeee62448.slice/crio-d94573e34d6a6969a95d1141f060bf93a358160c7e7009cfa698f6411a5267fb WatchSource:0}: Error finding container d94573e34d6a6969a95d1141f060bf93a358160c7e7009cfa698f6411a5267fb: Status 404 returned error can't find the container with id d94573e34d6a6969a95d1141f060bf93a358160c7e7009cfa698f6411a5267fb
Oct 30 13:11:20.224985 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:20.224929    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-xs2vr" event={"ID":"e3af0d52-346e-48d7-ac7d-8f2aeee62448","Type":"ContainerStarted","Data":"d94573e34d6a6969a95d1141f060bf93a358160c7e7009cfa698f6411a5267fb"}
Oct 30 13:11:21.824595 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:21.824561    4943 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 13:11:21.824904 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:21.824629    4943 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 13:11:21.824904 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:21.824658    4943 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 13:11:22.244283 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:22.244215    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-xs2vr" event={"ID":"e3af0d52-346e-48d7-ac7d-8f2aeee62448","Type":"ContainerStarted","Data":"0033fcc20732bb67a1bf7c2e2cf5e50511434b88f23ad93d8bff99cde7b389c6"}
Oct 30 13:11:22.311023 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:22.310856    4943 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-xs2vr" podStartSLOduration=1.8428074049999998 podStartE2EDuration="4.310837117s" totalImagesPullingTime="2.468029712s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:11:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:11:22.310755782 +0000 UTC m=+858.098759511" watchObservedRunningTime="2026-10-30 13:11:22.310837117 +0000 UTC m=+858.098840847"
Oct 30 13:11:23.036754 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:23.036718    4943 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-xs2vr"
Oct 30 13:11:23.038269 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:23.038235    4943 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-c9ffffc7f-xs2vr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" start-of-body=
Oct 30 13:11:23.038328 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:23.038310    4943 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-xs2vr" podUID="e3af0d52-346e-48d7-ac7d-8f2aeee62448" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused"
Oct 30 13:11:24.036910 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:24.036870    4943 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-c9ffffc7f-xs2vr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" start-of-body=
Oct 30 13:11:24.037318 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:24.036929    4943 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-xs2vr" podUID="e3af0d52-346e-48d7-ac7d-8f2aeee62448" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused"
Oct 30 13:11:25.037249 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:25.037215    4943 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-c9ffffc7f-xs2vr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" start-of-body=
Oct 30 13:11:25.037610 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:25.037282    4943 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-xs2vr" podUID="e3af0d52-346e-48d7-ac7d-8f2aeee62448" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused"
Oct 30 13:11:26.036409 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:26.036368    4943 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-c9ffffc7f-xs2vr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" start-of-body=
Oct 30 13:11:26.036616 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:26.036442    4943 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-xs2vr" podUID="e3af0d52-346e-48d7-ac7d-8f2aeee62448" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused"
Oct 30 13:11:27.037200 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:27.037167    4943 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-c9ffffc7f-xs2vr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" start-of-body=
Oct 30 13:11:27.037632 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:27.037221    4943 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-xs2vr" podUID="e3af0d52-346e-48d7-ac7d-8f2aeee62448" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused"
Oct 30 13:11:28.037214 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:28.037180    4943 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-c9ffffc7f-xs2vr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" start-of-body=
Oct 30 13:11:28.037646 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:28.037236    4943 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-xs2vr" podUID="e3af0d52-346e-48d7-ac7d-8f2aeee62448" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused"
Oct 30 13:11:29.036933 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:29.036895    4943 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-c9ffffc7f-xs2vr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" start-of-body=
Oct 30 13:11:29.037100 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:29.036955    4943 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-xs2vr" podUID="e3af0d52-346e-48d7-ac7d-8f2aeee62448" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused"
Oct 30 13:11:29.196442 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:29.196406    4943 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-78df7c7756-pzxsg"]
Oct 30 13:11:29.200188 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:29.200164    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-78df7c7756-pzxsg"
Oct 30 13:11:29.222682 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:29.222655    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-78df7c7756-pzxsg"]
Oct 30 13:11:29.344204 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:29.344112    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rt7hp\" (UniqueName: \"kubernetes.io/projected/bba72dc9-d1a2-4cf5-a205-4ee7250c7916-kube-api-access-rt7hp\") pod \"authorino-78df7c7756-pzxsg\" (UID: \"bba72dc9-d1a2-4cf5-a205-4ee7250c7916\") " pod="kuadrant-system/authorino-78df7c7756-pzxsg"
Oct 30 13:11:29.344204 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:29.344172    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/bba72dc9-d1a2-4cf5-a205-4ee7250c7916-tls-cert\") pod \"authorino-78df7c7756-pzxsg\" (UID: \"bba72dc9-d1a2-4cf5-a205-4ee7250c7916\") " pod="kuadrant-system/authorino-78df7c7756-pzxsg"
Oct 30 13:11:29.445295 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:29.445238    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/bba72dc9-d1a2-4cf5-a205-4ee7250c7916-tls-cert\") pod \"authorino-78df7c7756-pzxsg\" (UID: \"bba72dc9-d1a2-4cf5-a205-4ee7250c7916\") " pod="kuadrant-system/authorino-78df7c7756-pzxsg"
Oct 30 13:11:29.445492 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:29.445354    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rt7hp\" (UniqueName: \"kubernetes.io/projected/bba72dc9-d1a2-4cf5-a205-4ee7250c7916-kube-api-access-rt7hp\") pod \"authorino-78df7c7756-pzxsg\" (UID: \"bba72dc9-d1a2-4cf5-a205-4ee7250c7916\") " pod="kuadrant-system/authorino-78df7c7756-pzxsg"
Oct 30 13:11:29.447773 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:29.447745    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/bba72dc9-d1a2-4cf5-a205-4ee7250c7916-tls-cert\") pod \"authorino-78df7c7756-pzxsg\" (UID: \"bba72dc9-d1a2-4cf5-a205-4ee7250c7916\") " pod="kuadrant-system/authorino-78df7c7756-pzxsg"
Oct 30 13:11:29.484020 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:29.483992    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rt7hp\" (UniqueName: \"kubernetes.io/projected/bba72dc9-d1a2-4cf5-a205-4ee7250c7916-kube-api-access-rt7hp\") pod \"authorino-78df7c7756-pzxsg\" (UID: \"bba72dc9-d1a2-4cf5-a205-4ee7250c7916\") " pod="kuadrant-system/authorino-78df7c7756-pzxsg"
Oct 30 13:11:29.513065 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:29.513031    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-78df7c7756-pzxsg"
Oct 30 13:11:29.820634 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:29.820598    4943 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/ai-gateway-controller-697b76b867-4s264"]
Oct 30 13:11:29.825494 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:29.825473    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/ai-gateway-controller-697b76b867-4s264"
Oct 30 13:11:29.841604 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:29.841583    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-78df7c7756-pzxsg"]
Oct 30 13:11:29.844187 ip-10-0-142-84 kubenswrapper[4943]: W1030 13:11:29.844160    4943 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbba72dc9_d1a2_4cf5_a205_4ee7250c7916.slice/crio-e2df5129ba274edd68e55fdc43c14cb875d0e319ae1bed692eb6f7075ec2e3e4 WatchSource:0}: Error finding container e2df5129ba274edd68e55fdc43c14cb875d0e319ae1bed692eb6f7075ec2e3e4: Status 404 returned error can't find the container with id e2df5129ba274edd68e55fdc43c14cb875d0e319ae1bed692eb6f7075ec2e3e4
Oct 30 13:11:29.853355 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:29.850376    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/ai-gateway-controller-697b76b867-4s264"]
Oct 30 13:11:29.854831 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:29.854813    4943 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"ai-gateway-controller-parameters\""
Oct 30 13:11:29.854911 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:29.854904    4943 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"ai-gateway-controller-dockercfg-bpfnq\""
Oct 30 13:11:29.860837 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:29.860817    4943 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"ai-gateway-controller-metrics-tls\""
Oct 30 13:11:29.949389 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:29.949356    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hjnqj\" (UniqueName: \"kubernetes.io/projected/40dbf3f7-ccf2-4e8e-b5fb-7ea4e1fc42fd-kube-api-access-hjnqj\") pod \"ai-gateway-controller-697b76b867-4s264\" (UID: \"40dbf3f7-ccf2-4e8e-b5fb-7ea4e1fc42fd\") " pod="opendatahub/ai-gateway-controller-697b76b867-4s264"
Oct 30 13:11:29.949562 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:29.949428    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/40dbf3f7-ccf2-4e8e-b5fb-7ea4e1fc42fd-metrics-certs\") pod \"ai-gateway-controller-697b76b867-4s264\" (UID: \"40dbf3f7-ccf2-4e8e-b5fb-7ea4e1fc42fd\") " pod="opendatahub/ai-gateway-controller-697b76b867-4s264"
Oct 30 13:11:30.050439 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:30.050400    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/40dbf3f7-ccf2-4e8e-b5fb-7ea4e1fc42fd-metrics-certs\") pod \"ai-gateway-controller-697b76b867-4s264\" (UID: \"40dbf3f7-ccf2-4e8e-b5fb-7ea4e1fc42fd\") " pod="opendatahub/ai-gateway-controller-697b76b867-4s264"
Oct 30 13:11:30.050636 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:30.050497    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hjnqj\" (UniqueName: \"kubernetes.io/projected/40dbf3f7-ccf2-4e8e-b5fb-7ea4e1fc42fd-kube-api-access-hjnqj\") pod \"ai-gateway-controller-697b76b867-4s264\" (UID: \"40dbf3f7-ccf2-4e8e-b5fb-7ea4e1fc42fd\") " pod="opendatahub/ai-gateway-controller-697b76b867-4s264"
Oct 30 13:11:30.052890 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:30.052863    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/40dbf3f7-ccf2-4e8e-b5fb-7ea4e1fc42fd-metrics-certs\") pod \"ai-gateway-controller-697b76b867-4s264\" (UID: \"40dbf3f7-ccf2-4e8e-b5fb-7ea4e1fc42fd\") " pod="opendatahub/ai-gateway-controller-697b76b867-4s264"
Oct 30 13:11:30.076184 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:30.076131    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hjnqj\" (UniqueName: \"kubernetes.io/projected/40dbf3f7-ccf2-4e8e-b5fb-7ea4e1fc42fd-kube-api-access-hjnqj\") pod \"ai-gateway-controller-697b76b867-4s264\" (UID: \"40dbf3f7-ccf2-4e8e-b5fb-7ea4e1fc42fd\") " pod="opendatahub/ai-gateway-controller-697b76b867-4s264"
Oct 30 13:11:30.127693 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:30.127657    4943 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-xs2vr"
Oct 30 13:11:30.128007 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:30.127980    4943 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-xs2vr"
Oct 30 13:11:30.128881 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:30.128861    4943 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-xs2vr"
Oct 30 13:11:30.139049 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:30.139029    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/ai-gateway-controller-697b76b867-4s264"
Oct 30 13:11:30.309598 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:30.309565    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-78df7c7756-pzxsg" event={"ID":"bba72dc9-d1a2-4cf5-a205-4ee7250c7916","Type":"ContainerStarted","Data":"e2df5129ba274edd68e55fdc43c14cb875d0e319ae1bed692eb6f7075ec2e3e4"}
Oct 30 13:11:30.451778 ip-10-0-142-84 kubenswrapper[4943]: W1030 13:11:30.451742    4943 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod40dbf3f7_ccf2_4e8e_b5fb_7ea4e1fc42fd.slice/crio-0a36ca8834121d52e2cb5c0e860abcfe4ce0f50de407c8b22639391ef67aa49f WatchSource:0}: Error finding container 0a36ca8834121d52e2cb5c0e860abcfe4ce0f50de407c8b22639391ef67aa49f: Status 404 returned error can't find the container with id 0a36ca8834121d52e2cb5c0e860abcfe4ce0f50de407c8b22639391ef67aa49f
Oct 30 13:11:30.452094 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:30.452069    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/ai-gateway-controller-697b76b867-4s264"]
Oct 30 13:11:31.322167 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:31.322131    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/ai-gateway-controller-697b76b867-4s264" event={"ID":"40dbf3f7-ccf2-4e8e-b5fb-7ea4e1fc42fd","Type":"ContainerStarted","Data":"0a36ca8834121d52e2cb5c0e860abcfe4ce0f50de407c8b22639391ef67aa49f"}
Oct 30 13:11:31.325687 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:31.325572    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-78df7c7756-pzxsg" event={"ID":"bba72dc9-d1a2-4cf5-a205-4ee7250c7916","Type":"ContainerStarted","Data":"239d7dae427301f7c7226229ad005f10d9fa0865fb1c2c98fb12d0b3791a5f3c"}
Oct 30 13:11:31.375142 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:31.374779    4943 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/authorino-78df7c7756-pzxsg" podStartSLOduration=1.856177092 podStartE2EDuration="2.374759918s" totalImagesPullingTime="518.582826ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:11:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:11:31.371769476 +0000 UTC m=+867.159773231" watchObservedRunningTime="2026-10-30 13:11:31.374759918 +0000 UTC m=+867.162763649"
Oct 30 13:11:31.419288 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:31.416463    4943 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-65d74d5fff-gstt2"]
Oct 30 13:11:31.419288 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:31.416656    4943 kuberuntime_container.go:898] "Killing container with a grace period" pod="kuadrant-system/authorino-65d74d5fff-gstt2" podUID="be5a1e53-cca7-4305-a1dc-6882e6cd5b60" containerName="authorino" containerID="cri-o://fb8667c69c4d108704b3d09ecdc9bb95f87ecd65f10f69b9bdb6944795f2418f" gracePeriod=30
Oct 30 13:11:31.731794 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:31.731764    4943 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-65d74d5fff-gstt2"
Oct 30 13:11:31.871461 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:31.871425    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/be5a1e53-cca7-4305-a1dc-6882e6cd5b60-kube-api-access-pstmt\" (UniqueName: \"kubernetes.io/projected/be5a1e53-cca7-4305-a1dc-6882e6cd5b60-kube-api-access-pstmt\") pod \"be5a1e53-cca7-4305-a1dc-6882e6cd5b60\" (UID: \"be5a1e53-cca7-4305-a1dc-6882e6cd5b60\") "
Oct 30 13:11:31.871614 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:31.871582    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/be5a1e53-cca7-4305-a1dc-6882e6cd5b60-tls-cert\" (UniqueName: \"kubernetes.io/secret/be5a1e53-cca7-4305-a1dc-6882e6cd5b60-tls-cert\") pod \"be5a1e53-cca7-4305-a1dc-6882e6cd5b60\" (UID: \"be5a1e53-cca7-4305-a1dc-6882e6cd5b60\") "
Oct 30 13:11:31.873804 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:31.873771    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/be5a1e53-cca7-4305-a1dc-6882e6cd5b60-kube-api-access-pstmt" pod "be5a1e53-cca7-4305-a1dc-6882e6cd5b60" (UID: "be5a1e53-cca7-4305-a1dc-6882e6cd5b60"). InnerVolumeSpecName "kube-api-access-pstmt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:11:31.885719 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:31.885661    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/be5a1e53-cca7-4305-a1dc-6882e6cd5b60-tls-cert" pod "be5a1e53-cca7-4305-a1dc-6882e6cd5b60" (UID: "be5a1e53-cca7-4305-a1dc-6882e6cd5b60"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 13:11:31.972634 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:31.972585    4943 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/be5a1e53-cca7-4305-a1dc-6882e6cd5b60-tls-cert\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:11:31.972634 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:31.972628    4943 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pstmt\" (UniqueName: \"kubernetes.io/projected/be5a1e53-cca7-4305-a1dc-6882e6cd5b60-kube-api-access-pstmt\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:11:32.332874 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:32.332831    4943 generic.go:356] "Generic (PLEG): container finished" podID="be5a1e53-cca7-4305-a1dc-6882e6cd5b60" containerID="fb8667c69c4d108704b3d09ecdc9bb95f87ecd65f10f69b9bdb6944795f2418f" exitCode=0
Oct 30 13:11:32.333325 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:32.332896    4943 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-65d74d5fff-gstt2"
Oct 30 13:11:32.333325 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:32.332916    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-65d74d5fff-gstt2" event={"ID":"be5a1e53-cca7-4305-a1dc-6882e6cd5b60","Type":"ContainerDied","Data":"fb8667c69c4d108704b3d09ecdc9bb95f87ecd65f10f69b9bdb6944795f2418f"}
Oct 30 13:11:32.333325 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:32.332956    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-65d74d5fff-gstt2" event={"ID":"be5a1e53-cca7-4305-a1dc-6882e6cd5b60","Type":"ContainerDied","Data":"152d92d7a41eb2904336e66c609b252cae6862a183da9525700581ed89452153"}
Oct 30 13:11:32.333325 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:32.332978    4943 scope.go:122] "RemoveContainer" containerID="fb8667c69c4d108704b3d09ecdc9bb95f87ecd65f10f69b9bdb6944795f2418f"
Oct 30 13:11:32.345173 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:32.345152    4943 scope.go:122] "RemoveContainer" containerID="fb8667c69c4d108704b3d09ecdc9bb95f87ecd65f10f69b9bdb6944795f2418f"
Oct 30 13:11:32.345542 ip-10-0-142-84 kubenswrapper[4943]: E1030 13:11:32.345513    4943 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fb8667c69c4d108704b3d09ecdc9bb95f87ecd65f10f69b9bdb6944795f2418f\": container with ID starting with fb8667c69c4d108704b3d09ecdc9bb95f87ecd65f10f69b9bdb6944795f2418f not found: ID does not exist" containerID="fb8667c69c4d108704b3d09ecdc9bb95f87ecd65f10f69b9bdb6944795f2418f"
Oct 30 13:11:32.345621 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:32.345554    4943 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fb8667c69c4d108704b3d09ecdc9bb95f87ecd65f10f69b9bdb6944795f2418f"} err="failed to get container status \"fb8667c69c4d108704b3d09ecdc9bb95f87ecd65f10f69b9bdb6944795f2418f\": rpc error: code = NotFound desc = could not find container \"fb8667c69c4d108704b3d09ecdc9bb95f87ecd65f10f69b9bdb6944795f2418f\": container with ID starting with fb8667c69c4d108704b3d09ecdc9bb95f87ecd65f10f69b9bdb6944795f2418f not found: ID does not exist"
Oct 30 13:11:32.392449 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:32.391768    4943 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-65d74d5fff-gstt2"]
Oct 30 13:11:32.405542 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:32.405512    4943 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-65d74d5fff-gstt2"]
Oct 30 13:11:33.342227 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:33.342189    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/ai-gateway-controller-697b76b867-4s264" event={"ID":"40dbf3f7-ccf2-4e8e-b5fb-7ea4e1fc42fd","Type":"ContainerStarted","Data":"d72afde53bf57fbe0aa7fed80adc687a3eacfd5ee60fdcf9b5862d4b514d0884"}
Oct 30 13:11:33.342601 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:33.342320    4943 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/ai-gateway-controller-697b76b867-4s264"
Oct 30 13:11:33.414504 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:33.414453    4943 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/ai-gateway-controller-697b76b867-4s264" podStartSLOduration=1.6470288659999999 podStartE2EDuration="4.414439332s" totalImagesPullingTime="2.767410466s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:11:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:11:33.399286113 +0000 UTC m=+869.187289837" watchObservedRunningTime="2026-10-30 13:11:33.414439332 +0000 UTC m=+869.202443061"
Oct 30 13:11:33.793878 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:33.793839    4943 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-q58qm"]
Oct 30 13:11:33.794488 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:33.794464    4943 cpu_manager.go:414] "containerMap: removing container" podUID="be5a1e53-cca7-4305-a1dc-6882e6cd5b60" containerName="authorino"
Oct 30 13:11:33.794531 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:33.794494    4943 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="be5a1e53-cca7-4305-a1dc-6882e6cd5b60" containerName="authorino"
Oct 30 13:11:33.794653 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:33.794640    4943 memory_manager.go:365] "RemoveStaleState removing state" podUID="be5a1e53-cca7-4305-a1dc-6882e6cd5b60" containerName="authorino"
Oct 30 13:11:33.798526 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:33.798503    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-q58qm"
Oct 30 13:11:33.834547 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:33.834512    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-q58qm"]
Oct 30 13:11:33.898913 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:33.898866    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/883f1e32-141f-45e4-9f0e-31f9231fab0d-credential-socket\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-q58qm\" (UID: \"883f1e32-141f-45e4-9f0e-31f9231fab0d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-q58qm"
Oct 30 13:11:33.899098 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:33.898923    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/883f1e32-141f-45e4-9f0e-31f9231fab0d-workload-certs\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-q58qm\" (UID: \"883f1e32-141f-45e4-9f0e-31f9231fab0d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-q58qm"
Oct 30 13:11:33.899098 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:33.898996    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/883f1e32-141f-45e4-9f0e-31f9231fab0d-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-q58qm\" (UID: \"883f1e32-141f-45e4-9f0e-31f9231fab0d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-q58qm"
Oct 30 13:11:33.899098 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:33.899058    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gx7cn\" (UniqueName: \"kubernetes.io/projected/883f1e32-141f-45e4-9f0e-31f9231fab0d-kube-api-access-gx7cn\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-q58qm\" (UID: \"883f1e32-141f-45e4-9f0e-31f9231fab0d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-q58qm"
Oct 30 13:11:33.899317 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:33.899109    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/883f1e32-141f-45e4-9f0e-31f9231fab0d-istio-podinfo\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-q58qm\" (UID: \"883f1e32-141f-45e4-9f0e-31f9231fab0d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-q58qm"
Oct 30 13:11:33.899317 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:33.899195    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/883f1e32-141f-45e4-9f0e-31f9231fab0d-istio-token\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-q58qm\" (UID: \"883f1e32-141f-45e4-9f0e-31f9231fab0d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-q58qm"
Oct 30 13:11:33.899317 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:33.899230    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/883f1e32-141f-45e4-9f0e-31f9231fab0d-workload-socket\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-q58qm\" (UID: \"883f1e32-141f-45e4-9f0e-31f9231fab0d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-q58qm"
Oct 30 13:11:33.899317 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:33.899251    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/883f1e32-141f-45e4-9f0e-31f9231fab0d-istio-envoy\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-q58qm\" (UID: \"883f1e32-141f-45e4-9f0e-31f9231fab0d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-q58qm"
Oct 30 13:11:33.899317 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:33.899306    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/883f1e32-141f-45e4-9f0e-31f9231fab0d-istio-data\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-q58qm\" (UID: \"883f1e32-141f-45e4-9f0e-31f9231fab0d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-q58qm"
Oct 30 13:11:34.000412 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:34.000374    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/883f1e32-141f-45e4-9f0e-31f9231fab0d-istio-token\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-q58qm\" (UID: \"883f1e32-141f-45e4-9f0e-31f9231fab0d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-q58qm"
Oct 30 13:11:34.000584 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:34.000432    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/883f1e32-141f-45e4-9f0e-31f9231fab0d-workload-socket\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-q58qm\" (UID: \"883f1e32-141f-45e4-9f0e-31f9231fab0d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-q58qm"
Oct 30 13:11:34.000584 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:34.000454    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/883f1e32-141f-45e4-9f0e-31f9231fab0d-istio-envoy\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-q58qm\" (UID: \"883f1e32-141f-45e4-9f0e-31f9231fab0d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-q58qm"
Oct 30 13:11:34.000584 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:34.000469    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/883f1e32-141f-45e4-9f0e-31f9231fab0d-istio-data\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-q58qm\" (UID: \"883f1e32-141f-45e4-9f0e-31f9231fab0d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-q58qm"
Oct 30 13:11:34.000584 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:34.000509    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/883f1e32-141f-45e4-9f0e-31f9231fab0d-credential-socket\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-q58qm\" (UID: \"883f1e32-141f-45e4-9f0e-31f9231fab0d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-q58qm"
Oct 30 13:11:34.000584 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:34.000549    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/883f1e32-141f-45e4-9f0e-31f9231fab0d-workload-certs\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-q58qm\" (UID: \"883f1e32-141f-45e4-9f0e-31f9231fab0d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-q58qm"
Oct 30 13:11:34.000584 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:34.000575    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/883f1e32-141f-45e4-9f0e-31f9231fab0d-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-q58qm\" (UID: \"883f1e32-141f-45e4-9f0e-31f9231fab0d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-q58qm"
Oct 30 13:11:34.000823 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:34.000600    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gx7cn\" (UniqueName: \"kubernetes.io/projected/883f1e32-141f-45e4-9f0e-31f9231fab0d-kube-api-access-gx7cn\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-q58qm\" (UID: \"883f1e32-141f-45e4-9f0e-31f9231fab0d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-q58qm"
Oct 30 13:11:34.000823 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:34.000654    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/883f1e32-141f-45e4-9f0e-31f9231fab0d-istio-podinfo\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-q58qm\" (UID: \"883f1e32-141f-45e4-9f0e-31f9231fab0d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-q58qm"
Oct 30 13:11:34.000950 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:34.000921    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/883f1e32-141f-45e4-9f0e-31f9231fab0d-workload-socket\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-q58qm\" (UID: \"883f1e32-141f-45e4-9f0e-31f9231fab0d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-q58qm"
Oct 30 13:11:34.001015 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:34.000989    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/883f1e32-141f-45e4-9f0e-31f9231fab0d-istio-data\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-q58qm\" (UID: \"883f1e32-141f-45e4-9f0e-31f9231fab0d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-q58qm"
Oct 30 13:11:34.001210 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:34.001178    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/883f1e32-141f-45e4-9f0e-31f9231fab0d-workload-certs\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-q58qm\" (UID: \"883f1e32-141f-45e4-9f0e-31f9231fab0d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-q58qm"
Oct 30 13:11:34.001405 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:34.001277    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/883f1e32-141f-45e4-9f0e-31f9231fab0d-credential-socket\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-q58qm\" (UID: \"883f1e32-141f-45e4-9f0e-31f9231fab0d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-q58qm"
Oct 30 13:11:34.001640 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:34.001618    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/883f1e32-141f-45e4-9f0e-31f9231fab0d-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-q58qm\" (UID: \"883f1e32-141f-45e4-9f0e-31f9231fab0d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-q58qm"
Oct 30 13:11:34.003071 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:34.003049    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/883f1e32-141f-45e4-9f0e-31f9231fab0d-istio-envoy\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-q58qm\" (UID: \"883f1e32-141f-45e4-9f0e-31f9231fab0d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-q58qm"
Oct 30 13:11:34.003161 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:34.003108    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/883f1e32-141f-45e4-9f0e-31f9231fab0d-istio-podinfo\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-q58qm\" (UID: \"883f1e32-141f-45e4-9f0e-31f9231fab0d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-q58qm"
Oct 30 13:11:34.026204 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:34.026177    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/883f1e32-141f-45e4-9f0e-31f9231fab0d-istio-token\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-q58qm\" (UID: \"883f1e32-141f-45e4-9f0e-31f9231fab0d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-q58qm"
Oct 30 13:11:34.027840 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:34.027817    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gx7cn\" (UniqueName: \"kubernetes.io/projected/883f1e32-141f-45e4-9f0e-31f9231fab0d-kube-api-access-gx7cn\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-q58qm\" (UID: \"883f1e32-141f-45e4-9f0e-31f9231fab0d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-q58qm"
Oct 30 13:11:34.112708 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:34.112629    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-q58qm"
Oct 30 13:11:34.367092 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:34.367002    4943 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="be5a1e53-cca7-4305-a1dc-6882e6cd5b60" path="/var/lib/kubelet/pods/be5a1e53-cca7-4305-a1dc-6882e6cd5b60/volumes"
Oct 30 13:11:34.440917 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:34.440882    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-q58qm"]
Oct 30 13:11:34.442215 ip-10-0-142-84 kubenswrapper[4943]: W1030 13:11:34.442187    4943 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod883f1e32_141f_45e4_9f0e_31f9231fab0d.slice/crio-06d4c642e24ddb4d221354f0b1445d546a1e981e08e6c6f08c5eb3bcdd2abe80 WatchSource:0}: Error finding container 06d4c642e24ddb4d221354f0b1445d546a1e981e08e6c6f08c5eb3bcdd2abe80: Status 404 returned error can't find the container with id 06d4c642e24ddb4d221354f0b1445d546a1e981e08e6c6f08c5eb3bcdd2abe80
Oct 30 13:11:34.445967 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:34.445926    4943 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 13:11:34.446065 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:34.446019    4943 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 13:11:34.446065 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:34.446055    4943 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 13:11:35.398952 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:35.398911    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-q58qm" event={"ID":"883f1e32-141f-45e4-9f0e-31f9231fab0d","Type":"ContainerStarted","Data":"6c1e5ecd734fa6354f717ef54dd3d773e0452a91a0e6fba515a37fea89df216a"}
Oct 30 13:11:35.399562 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:35.399534    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-q58qm" event={"ID":"883f1e32-141f-45e4-9f0e-31f9231fab0d","Type":"ContainerStarted","Data":"06d4c642e24ddb4d221354f0b1445d546a1e981e08e6c6f08c5eb3bcdd2abe80"}
Oct 30 13:11:35.498986 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:35.498907    4943 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-q58qm" podStartSLOduration=2.498886568 podStartE2EDuration="2.498886568s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:11:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:11:35.47802517 +0000 UTC m=+871.266028902" watchObservedRunningTime="2026-10-30 13:11:35.498886568 +0000 UTC m=+871.286890299"
Oct 30 13:11:35.634178 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:35.634143    4943 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-7b98bb678c-666x8"]
Oct 30 13:11:35.638376 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:35.638346    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7b98bb678c-666x8"
Oct 30 13:11:35.660179 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:35.660108    4943 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-n4kpp\""
Oct 30 13:11:35.677290 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:35.661670    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7b98bb678c-666x8"]
Oct 30 13:11:35.723153 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:35.723112    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p92xk\" (UniqueName: \"kubernetes.io/projected/94a883ed-7c63-402d-bf2e-2e4ce2383175-kube-api-access-p92xk\") pod \"payload-pre-processing-7b98bb678c-666x8\" (UID: \"94a883ed-7c63-402d-bf2e-2e4ce2383175\") " pod="openshift-ingress/payload-pre-processing-7b98bb678c-666x8"
Oct 30 13:11:35.723733 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:35.723693    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/94a883ed-7c63-402d-bf2e-2e4ce2383175-plugins-config-volume\") pod \"payload-pre-processing-7b98bb678c-666x8\" (UID: \"94a883ed-7c63-402d-bf2e-2e4ce2383175\") " pod="openshift-ingress/payload-pre-processing-7b98bb678c-666x8"
Oct 30 13:11:35.725307 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:35.725285    4943 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Oct 30 13:11:35.743535 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:35.743496    4943 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-5f78687d5b-r2pcb"]
Oct 30 13:11:35.747409 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:35.747386    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5f78687d5b-r2pcb"
Oct 30 13:11:35.800143 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:35.800102    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5f78687d5b-r2pcb"]
Oct 30 13:11:35.824419 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:35.824384    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7d0643d5-c375-42a2-ab82-f90aa05ec22d-plugins-config-volume\") pod \"payload-processing-5f78687d5b-r2pcb\" (UID: \"7d0643d5-c375-42a2-ab82-f90aa05ec22d\") " pod="openshift-ingress/payload-processing-5f78687d5b-r2pcb"
Oct 30 13:11:35.824604 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:35.824448    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p92xk\" (UniqueName: \"kubernetes.io/projected/94a883ed-7c63-402d-bf2e-2e4ce2383175-kube-api-access-p92xk\") pod \"payload-pre-processing-7b98bb678c-666x8\" (UID: \"94a883ed-7c63-402d-bf2e-2e4ce2383175\") " pod="openshift-ingress/payload-pre-processing-7b98bb678c-666x8"
Oct 30 13:11:35.824604 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:35.824481    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wrpbq\" (UniqueName: \"kubernetes.io/projected/7d0643d5-c375-42a2-ab82-f90aa05ec22d-kube-api-access-wrpbq\") pod \"payload-processing-5f78687d5b-r2pcb\" (UID: \"7d0643d5-c375-42a2-ab82-f90aa05ec22d\") " pod="openshift-ingress/payload-processing-5f78687d5b-r2pcb"
Oct 30 13:11:35.824604 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:35.824513    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/94a883ed-7c63-402d-bf2e-2e4ce2383175-plugins-config-volume\") pod \"payload-pre-processing-7b98bb678c-666x8\" (UID: \"94a883ed-7c63-402d-bf2e-2e4ce2383175\") " pod="openshift-ingress/payload-pre-processing-7b98bb678c-666x8"
Oct 30 13:11:35.825206 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:35.825183    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/94a883ed-7c63-402d-bf2e-2e4ce2383175-plugins-config-volume\") pod \"payload-pre-processing-7b98bb678c-666x8\" (UID: \"94a883ed-7c63-402d-bf2e-2e4ce2383175\") " pod="openshift-ingress/payload-pre-processing-7b98bb678c-666x8"
Oct 30 13:11:35.850721 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:35.850682    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p92xk\" (UniqueName: \"kubernetes.io/projected/94a883ed-7c63-402d-bf2e-2e4ce2383175-kube-api-access-p92xk\") pod \"payload-pre-processing-7b98bb678c-666x8\" (UID: \"94a883ed-7c63-402d-bf2e-2e4ce2383175\") " pod="openshift-ingress/payload-pre-processing-7b98bb678c-666x8"
Oct 30 13:11:35.925931 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:35.925841    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wrpbq\" (UniqueName: \"kubernetes.io/projected/7d0643d5-c375-42a2-ab82-f90aa05ec22d-kube-api-access-wrpbq\") pod \"payload-processing-5f78687d5b-r2pcb\" (UID: \"7d0643d5-c375-42a2-ab82-f90aa05ec22d\") " pod="openshift-ingress/payload-processing-5f78687d5b-r2pcb"
Oct 30 13:11:35.925931 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:35.925932    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7d0643d5-c375-42a2-ab82-f90aa05ec22d-plugins-config-volume\") pod \"payload-processing-5f78687d5b-r2pcb\" (UID: \"7d0643d5-c375-42a2-ab82-f90aa05ec22d\") " pod="openshift-ingress/payload-processing-5f78687d5b-r2pcb"
Oct 30 13:11:35.926625 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:35.926604    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7d0643d5-c375-42a2-ab82-f90aa05ec22d-plugins-config-volume\") pod \"payload-processing-5f78687d5b-r2pcb\" (UID: \"7d0643d5-c375-42a2-ab82-f90aa05ec22d\") " pod="openshift-ingress/payload-processing-5f78687d5b-r2pcb"
Oct 30 13:11:35.952343 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:35.952306    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7b98bb678c-666x8"
Oct 30 13:11:35.998769 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:35.957349    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wrpbq\" (UniqueName: \"kubernetes.io/projected/7d0643d5-c375-42a2-ab82-f90aa05ec22d-kube-api-access-wrpbq\") pod \"payload-processing-5f78687d5b-r2pcb\" (UID: \"7d0643d5-c375-42a2-ab82-f90aa05ec22d\") " pod="openshift-ingress/payload-processing-5f78687d5b-r2pcb"
Oct 30 13:11:35.998769 ip-10-0-142-84 kubenswrapper[4943]: E1030 13:11:35.968835    4943 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-knqz6" containerName="extractor"
Oct 30 13:11:36.101561 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:36.101436    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5f78687d5b-r2pcb"
Oct 30 13:11:36.113598 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:36.113564    4943 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-q58qm"
Oct 30 13:11:36.114822 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:36.114795    4943 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-c9ffffc7f-q58qm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body=
Oct 30 13:11:36.114937 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:36.114847    4943 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-q58qm" podUID="883f1e32-141f-45e4-9f0e-31f9231fab0d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused"
Oct 30 13:11:36.309399 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:36.309367    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7b98bb678c-666x8"]
Oct 30 13:11:36.311359 ip-10-0-142-84 kubenswrapper[4943]: W1030 13:11:36.311329    4943 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod94a883ed_7c63_402d_bf2e_2e4ce2383175.slice/crio-da262471682ded62630d0131b921fda93719000a1323cb6ce75c5c17e86c76ef WatchSource:0}: Error finding container da262471682ded62630d0131b921fda93719000a1323cb6ce75c5c17e86c76ef: Status 404 returned error can't find the container with id da262471682ded62630d0131b921fda93719000a1323cb6ce75c5c17e86c76ef
Oct 30 13:11:36.406371 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:36.406331    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7b98bb678c-666x8" event={"ID":"94a883ed-7c63-402d-bf2e-2e4ce2383175","Type":"ContainerStarted","Data":"da262471682ded62630d0131b921fda93719000a1323cb6ce75c5c17e86c76ef"}
Oct 30 13:11:36.443219 ip-10-0-142-84 kubenswrapper[4943]: W1030 13:11:36.443179    4943 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7d0643d5_c375_42a2_ab82_f90aa05ec22d.slice/crio-9830d4b4ef01e3ac2aba10bb7649f3de49c85cefd92216bb4121925983e13c8f WatchSource:0}: Error finding container 9830d4b4ef01e3ac2aba10bb7649f3de49c85cefd92216bb4121925983e13c8f: Status 404 returned error can't find the container with id 9830d4b4ef01e3ac2aba10bb7649f3de49c85cefd92216bb4121925983e13c8f
Oct 30 13:11:36.447647 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:36.447623    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5f78687d5b-r2pcb"]
Oct 30 13:11:37.114422 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:37.113940    4943 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-c9ffffc7f-q58qm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body=
Oct 30 13:11:37.114422 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:37.114003    4943 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-q58qm" podUID="883f1e32-141f-45e4-9f0e-31f9231fab0d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused"
Oct 30 13:11:37.417888 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:37.417755    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5f78687d5b-r2pcb" event={"ID":"7d0643d5-c375-42a2-ab82-f90aa05ec22d","Type":"ContainerStarted","Data":"9830d4b4ef01e3ac2aba10bb7649f3de49c85cefd92216bb4121925983e13c8f"}
Oct 30 13:11:38.114169 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:38.113718    4943 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-c9ffffc7f-q58qm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body=
Oct 30 13:11:38.114169 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:38.113783    4943 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-q58qm" podUID="883f1e32-141f-45e4-9f0e-31f9231fab0d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused"
Oct 30 13:11:39.113377 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:39.113340    4943 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-c9ffffc7f-q58qm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body=
Oct 30 13:11:39.113873 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:39.113403    4943 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-q58qm" podUID="883f1e32-141f-45e4-9f0e-31f9231fab0d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused"
Oct 30 13:11:40.113103 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:40.113071    4943 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-c9ffffc7f-q58qm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body=
Oct 30 13:11:40.113274 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:40.113118    4943 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-q58qm" podUID="883f1e32-141f-45e4-9f0e-31f9231fab0d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused"
Oct 30 13:11:40.440641 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:40.440602    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5f78687d5b-r2pcb" event={"ID":"7d0643d5-c375-42a2-ab82-f90aa05ec22d","Type":"ContainerStarted","Data":"7a841c74a6c35dcae96284bd4ecbb1f43b7441eaa0e90c461c672a6c269a7c5d"}
Oct 30 13:11:40.441102 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:40.440716    4943 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-5f78687d5b-r2pcb"
Oct 30 13:11:40.442305 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:40.442279    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7b98bb678c-666x8" event={"ID":"94a883ed-7c63-402d-bf2e-2e4ce2383175","Type":"ContainerStarted","Data":"95daf0eaa5bc6778353b3158f759400a6efbf3cf095a1e58a2b5da0398bbb304"}
Oct 30 13:11:40.442424 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:40.442331    4943 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-7b98bb678c-666x8"
Oct 30 13:11:40.487911 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:40.487862    4943 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-5f78687d5b-r2pcb" podStartSLOduration=2.026991502 podStartE2EDuration="5.487850231s" totalImagesPullingTime="3.460858729s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:11:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:11:40.485939794 +0000 UTC m=+876.273943537" watchObservedRunningTime="2026-10-30 13:11:40.487850231 +0000 UTC m=+876.275853957"
Oct 30 13:11:40.534538 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:40.534494    4943 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-7b98bb678c-666x8" podStartSLOduration=1.8435738750000001 podStartE2EDuration="5.534481553s" totalImagesPullingTime="3.690907678s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:11:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:11:40.533122012 +0000 UTC m=+876.321125744" watchObservedRunningTime="2026-10-30 13:11:40.534481553 +0000 UTC m=+876.322485282"
Oct 30 13:11:41.113300 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:41.113243    4943 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-c9ffffc7f-q58qm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body=
Oct 30 13:11:41.113473 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:41.113325    4943 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-q58qm" podUID="883f1e32-141f-45e4-9f0e-31f9231fab0d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused"
Oct 30 13:11:43.113761 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:43.113675    4943 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-c9ffffc7f-q58qm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 13:11:43.114199 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:43.113796    4943 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-q58qm" podUID="883f1e32-141f-45e4-9f0e-31f9231fab0d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:11:44.155413 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:44.155375    4943 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-c9ffffc7f-q58qm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 13:11:44.155802 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:44.155443    4943 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-q58qm" podUID="883f1e32-141f-45e4-9f0e-31f9231fab0d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:11:44.370301 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:44.370252    4943 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/ai-gateway-controller-697b76b867-4s264"
Oct 30 13:11:44.741462 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:44.741425    4943 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/ai-gateway-controller-976c7b7d8-tlndh"]
Oct 30 13:11:44.745213 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:44.745195    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/ai-gateway-controller-976c7b7d8-tlndh"
Oct 30 13:11:44.770897 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:44.770867    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/ai-gateway-controller-976c7b7d8-tlndh"]
Oct 30 13:11:44.833244 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:44.833204    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hhlpn\" (UniqueName: \"kubernetes.io/projected/5c697467-45ca-4c0e-88e4-32e23cb6c2dd-kube-api-access-hhlpn\") pod \"ai-gateway-controller-976c7b7d8-tlndh\" (UID: \"5c697467-45ca-4c0e-88e4-32e23cb6c2dd\") " pod="opendatahub/ai-gateway-controller-976c7b7d8-tlndh"
Oct 30 13:11:44.833442 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:44.833408    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5c697467-45ca-4c0e-88e4-32e23cb6c2dd-metrics-certs\") pod \"ai-gateway-controller-976c7b7d8-tlndh\" (UID: \"5c697467-45ca-4c0e-88e4-32e23cb6c2dd\") " pod="opendatahub/ai-gateway-controller-976c7b7d8-tlndh"
Oct 30 13:11:44.917661 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:44.917630    4943 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-q58qm"
Oct 30 13:11:44.917951 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:44.917930    4943 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-q58qm"
Oct 30 13:11:44.918662 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:44.918644    4943 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-q58qm"
Oct 30 13:11:44.934712 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:44.934686    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5c697467-45ca-4c0e-88e4-32e23cb6c2dd-metrics-certs\") pod \"ai-gateway-controller-976c7b7d8-tlndh\" (UID: \"5c697467-45ca-4c0e-88e4-32e23cb6c2dd\") " pod="opendatahub/ai-gateway-controller-976c7b7d8-tlndh"
Oct 30 13:11:44.934860 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:44.934739    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hhlpn\" (UniqueName: \"kubernetes.io/projected/5c697467-45ca-4c0e-88e4-32e23cb6c2dd-kube-api-access-hhlpn\") pod \"ai-gateway-controller-976c7b7d8-tlndh\" (UID: \"5c697467-45ca-4c0e-88e4-32e23cb6c2dd\") " pod="opendatahub/ai-gateway-controller-976c7b7d8-tlndh"
Oct 30 13:11:44.937074 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:44.937051    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5c697467-45ca-4c0e-88e4-32e23cb6c2dd-metrics-certs\") pod \"ai-gateway-controller-976c7b7d8-tlndh\" (UID: \"5c697467-45ca-4c0e-88e4-32e23cb6c2dd\") " pod="opendatahub/ai-gateway-controller-976c7b7d8-tlndh"
Oct 30 13:11:44.960829 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:44.960809    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hhlpn\" (UniqueName: \"kubernetes.io/projected/5c697467-45ca-4c0e-88e4-32e23cb6c2dd-kube-api-access-hhlpn\") pod \"ai-gateway-controller-976c7b7d8-tlndh\" (UID: \"5c697467-45ca-4c0e-88e4-32e23cb6c2dd\") " pod="opendatahub/ai-gateway-controller-976c7b7d8-tlndh"
Oct 30 13:11:45.058027 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:45.057948    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/ai-gateway-controller-976c7b7d8-tlndh"
Oct 30 13:11:45.362426 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:45.362389    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/ai-gateway-controller-976c7b7d8-tlndh"]
Oct 30 13:11:45.363910 ip-10-0-142-84 kubenswrapper[4943]: W1030 13:11:45.363886    4943 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5c697467_45ca_4c0e_88e4_32e23cb6c2dd.slice/crio-9405ad1bfbcea6abd416f090ba5f27de5b2c7af303f329f5c109750b04ce56c3 WatchSource:0}: Error finding container 9405ad1bfbcea6abd416f090ba5f27de5b2c7af303f329f5c109750b04ce56c3: Status 404 returned error can't find the container with id 9405ad1bfbcea6abd416f090ba5f27de5b2c7af303f329f5c109750b04ce56c3
Oct 30 13:11:45.471202 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:45.471168    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/ai-gateway-controller-976c7b7d8-tlndh" event={"ID":"5c697467-45ca-4c0e-88e4-32e23cb6c2dd","Type":"ContainerStarted","Data":"9405ad1bfbcea6abd416f090ba5f27de5b2c7af303f329f5c109750b04ce56c3"}
Oct 30 13:11:46.477077 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:46.476712    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/ai-gateway-controller-976c7b7d8-tlndh" event={"ID":"5c697467-45ca-4c0e-88e4-32e23cb6c2dd","Type":"ContainerStarted","Data":"79273f2ee62c5f9af1e52a84437d74514528cf21744cab75211fba249536bb8a"}
Oct 30 13:11:46.477564 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:46.477435    4943 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/ai-gateway-controller-976c7b7d8-tlndh"
Oct 30 13:11:46.522071 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:46.522020    4943 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/ai-gateway-controller-976c7b7d8-tlndh" podStartSLOduration=2.180303725 podStartE2EDuration="2.522004299s" totalImagesPullingTime="341.700574ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:11:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:11:46.521776836 +0000 UTC m=+882.309780566" watchObservedRunningTime="2026-10-30 13:11:46.522004299 +0000 UTC m=+882.310008029"
Oct 30 13:11:48.829591 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:48.829554    4943 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-mlh2m"]
Oct 30 13:11:48.833121 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:48.833100    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-mlh2m"
Oct 30 13:11:48.884887 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:48.884850    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-mlh2m"]
Oct 30 13:11:48.964396 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:48.964361    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1c74e310-6638-4b15-97d8-65854946086a-credential-socket\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-mlh2m\" (UID: \"1c74e310-6638-4b15-97d8-65854946086a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-mlh2m"
Oct 30 13:11:48.964540 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:48.964456    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1c74e310-6638-4b15-97d8-65854946086a-istio-envoy\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-mlh2m\" (UID: \"1c74e310-6638-4b15-97d8-65854946086a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-mlh2m"
Oct 30 13:11:48.964583 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:48.964557    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1c74e310-6638-4b15-97d8-65854946086a-workload-certs\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-mlh2m\" (UID: \"1c74e310-6638-4b15-97d8-65854946086a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-mlh2m"
Oct 30 13:11:48.964648 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:48.964629    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1c74e310-6638-4b15-97d8-65854946086a-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-mlh2m\" (UID: \"1c74e310-6638-4b15-97d8-65854946086a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-mlh2m"
Oct 30 13:11:48.964709 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:48.964687    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1c74e310-6638-4b15-97d8-65854946086a-istio-token\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-mlh2m\" (UID: \"1c74e310-6638-4b15-97d8-65854946086a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-mlh2m"
Oct 30 13:11:48.964770 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:48.964739    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1c74e310-6638-4b15-97d8-65854946086a-workload-socket\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-mlh2m\" (UID: \"1c74e310-6638-4b15-97d8-65854946086a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-mlh2m"
Oct 30 13:11:48.964850 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:48.964826    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5clq5\" (UniqueName: \"kubernetes.io/projected/1c74e310-6638-4b15-97d8-65854946086a-kube-api-access-5clq5\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-mlh2m\" (UID: \"1c74e310-6638-4b15-97d8-65854946086a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-mlh2m"
Oct 30 13:11:48.964923 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:48.964881    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1c74e310-6638-4b15-97d8-65854946086a-istio-podinfo\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-mlh2m\" (UID: \"1c74e310-6638-4b15-97d8-65854946086a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-mlh2m"
Oct 30 13:11:48.964983 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:48.964961    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1c74e310-6638-4b15-97d8-65854946086a-istio-data\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-mlh2m\" (UID: \"1c74e310-6638-4b15-97d8-65854946086a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-mlh2m"
Oct 30 13:11:49.065742 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:49.065712    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1c74e310-6638-4b15-97d8-65854946086a-workload-certs\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-mlh2m\" (UID: \"1c74e310-6638-4b15-97d8-65854946086a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-mlh2m"
Oct 30 13:11:49.065886 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:49.065758    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1c74e310-6638-4b15-97d8-65854946086a-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-mlh2m\" (UID: \"1c74e310-6638-4b15-97d8-65854946086a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-mlh2m"
Oct 30 13:11:49.065886 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:49.065779    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1c74e310-6638-4b15-97d8-65854946086a-istio-token\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-mlh2m\" (UID: \"1c74e310-6638-4b15-97d8-65854946086a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-mlh2m"
Oct 30 13:11:49.065886 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:49.065803    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1c74e310-6638-4b15-97d8-65854946086a-workload-socket\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-mlh2m\" (UID: \"1c74e310-6638-4b15-97d8-65854946086a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-mlh2m"
Oct 30 13:11:49.065886 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:49.065828    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5clq5\" (UniqueName: \"kubernetes.io/projected/1c74e310-6638-4b15-97d8-65854946086a-kube-api-access-5clq5\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-mlh2m\" (UID: \"1c74e310-6638-4b15-97d8-65854946086a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-mlh2m"
Oct 30 13:11:49.065886 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:49.065856    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1c74e310-6638-4b15-97d8-65854946086a-istio-podinfo\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-mlh2m\" (UID: \"1c74e310-6638-4b15-97d8-65854946086a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-mlh2m"
Oct 30 13:11:49.065886 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:49.065885    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1c74e310-6638-4b15-97d8-65854946086a-istio-data\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-mlh2m\" (UID: \"1c74e310-6638-4b15-97d8-65854946086a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-mlh2m"
Oct 30 13:11:49.066202 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:49.065939    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1c74e310-6638-4b15-97d8-65854946086a-credential-socket\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-mlh2m\" (UID: \"1c74e310-6638-4b15-97d8-65854946086a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-mlh2m"
Oct 30 13:11:49.066202 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:49.065970    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1c74e310-6638-4b15-97d8-65854946086a-istio-envoy\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-mlh2m\" (UID: \"1c74e310-6638-4b15-97d8-65854946086a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-mlh2m"
Oct 30 13:11:49.066202 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:49.066085    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1c74e310-6638-4b15-97d8-65854946086a-workload-certs\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-mlh2m\" (UID: \"1c74e310-6638-4b15-97d8-65854946086a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-mlh2m"
Oct 30 13:11:49.066202 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:49.066149    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1c74e310-6638-4b15-97d8-65854946086a-workload-socket\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-mlh2m\" (UID: \"1c74e310-6638-4b15-97d8-65854946086a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-mlh2m"
Oct 30 13:11:49.066425 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:49.066373    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1c74e310-6638-4b15-97d8-65854946086a-istio-data\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-mlh2m\" (UID: \"1c74e310-6638-4b15-97d8-65854946086a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-mlh2m"
Oct 30 13:11:49.066425 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:49.066399    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1c74e310-6638-4b15-97d8-65854946086a-credential-socket\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-mlh2m\" (UID: \"1c74e310-6638-4b15-97d8-65854946086a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-mlh2m"
Oct 30 13:11:49.066622 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:49.066601    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1c74e310-6638-4b15-97d8-65854946086a-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-mlh2m\" (UID: \"1c74e310-6638-4b15-97d8-65854946086a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-mlh2m"
Oct 30 13:11:49.068239 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:49.068217    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1c74e310-6638-4b15-97d8-65854946086a-istio-envoy\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-mlh2m\" (UID: \"1c74e310-6638-4b15-97d8-65854946086a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-mlh2m"
Oct 30 13:11:49.068476 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:49.068457    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1c74e310-6638-4b15-97d8-65854946086a-istio-podinfo\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-mlh2m\" (UID: \"1c74e310-6638-4b15-97d8-65854946086a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-mlh2m"
Oct 30 13:11:49.109713 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:49.109659    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5clq5\" (UniqueName: \"kubernetes.io/projected/1c74e310-6638-4b15-97d8-65854946086a-kube-api-access-5clq5\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-mlh2m\" (UID: \"1c74e310-6638-4b15-97d8-65854946086a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-mlh2m"
Oct 30 13:11:49.109865 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:49.109830    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1c74e310-6638-4b15-97d8-65854946086a-istio-token\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-mlh2m\" (UID: \"1c74e310-6638-4b15-97d8-65854946086a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-mlh2m"
Oct 30 13:11:49.146555 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:49.146525    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-mlh2m"
Oct 30 13:11:49.469673 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:49.469650    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-mlh2m"]
Oct 30 13:11:49.471469 ip-10-0-142-84 kubenswrapper[4943]: W1030 13:11:49.471440    4943 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1c74e310_6638_4b15_97d8_65854946086a.slice/crio-d1dd8a63c5e7a1a8fb3d8ed87cc006c0f693bcc3da2ca5b56f1d4ea534058f91 WatchSource:0}: Error finding container d1dd8a63c5e7a1a8fb3d8ed87cc006c0f693bcc3da2ca5b56f1d4ea534058f91: Status 404 returned error can't find the container with id d1dd8a63c5e7a1a8fb3d8ed87cc006c0f693bcc3da2ca5b56f1d4ea534058f91
Oct 30 13:11:49.473542 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:49.473503    4943 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 13:11:49.473617 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:49.473576    4943 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 13:11:49.473617 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:49.473609    4943 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 13:11:49.493678 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:49.493653    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-mlh2m" event={"ID":"1c74e310-6638-4b15-97d8-65854946086a","Type":"ContainerStarted","Data":"d1dd8a63c5e7a1a8fb3d8ed87cc006c0f693bcc3da2ca5b56f1d4ea534058f91"}
Oct 30 13:11:50.500465 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:50.500428    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-mlh2m" event={"ID":"1c74e310-6638-4b15-97d8-65854946086a","Type":"ContainerStarted","Data":"343c2ddb8a87c1a0d9493df064e5f7fb86fe37d31cd4e3664dfeadb6b3cc133b"}
Oct 30 13:11:50.596098 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:50.596038    4943 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-mlh2m" podStartSLOduration=2.596024372 podStartE2EDuration="2.596024372s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:11:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:11:50.580744941 +0000 UTC m=+886.368748718" watchObservedRunningTime="2026-10-30 13:11:50.596024372 +0000 UTC m=+886.384028103"
Oct 30 13:11:51.147451 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:51.147405    4943 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-mlh2m"
Oct 30 13:11:51.149091 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:51.149068    4943 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-c9ffffc7f-mlh2m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body=
Oct 30 13:11:51.149162 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:51.149118    4943 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-mlh2m" podUID="1c74e310-6638-4b15-97d8-65854946086a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused"
Oct 30 13:11:51.450789 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:51.450757    4943 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-7b98bb678c-666x8"
Oct 30 13:11:51.450954 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:51.450828    4943 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-5f78687d5b-r2pcb"
Oct 30 13:11:52.147586 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:52.147550    4943 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-c9ffffc7f-mlh2m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body=
Oct 30 13:11:52.147978 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:52.147601    4943 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-mlh2m" podUID="1c74e310-6638-4b15-97d8-65854946086a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused"
Oct 30 13:11:53.147003 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:53.146965    4943 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-c9ffffc7f-mlh2m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body=
Oct 30 13:11:53.147209 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:53.147026    4943 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-mlh2m" podUID="1c74e310-6638-4b15-97d8-65854946086a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused"
Oct 30 13:11:54.147586 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:54.147548    4943 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-c9ffffc7f-mlh2m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body=
Oct 30 13:11:54.148068 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:54.147628    4943 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-mlh2m" podUID="1c74e310-6638-4b15-97d8-65854946086a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused"
Oct 30 13:11:55.147297 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:55.147244    4943 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-c9ffffc7f-mlh2m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body=
Oct 30 13:11:55.147477 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:55.147312    4943 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-mlh2m" podUID="1c74e310-6638-4b15-97d8-65854946086a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused"
Oct 30 13:11:56.147464 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:56.147415    4943 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-c9ffffc7f-mlh2m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body=
Oct 30 13:11:56.147870 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:56.147538    4943 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-mlh2m" podUID="1c74e310-6638-4b15-97d8-65854946086a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused"
Oct 30 13:11:57.484146 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:57.484113    4943 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/ai-gateway-controller-976c7b7d8-tlndh"
Oct 30 13:11:57.594489 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:57.594452    4943 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/ai-gateway-controller-697b76b867-4s264"]
Oct 30 13:11:57.594696 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:57.594659    4943 kuberuntime_container.go:898] "Killing container with a grace period" pod="opendatahub/ai-gateway-controller-697b76b867-4s264" podUID="40dbf3f7-ccf2-4e8e-b5fb-7ea4e1fc42fd" containerName="manager" containerID="cri-o://d72afde53bf57fbe0aa7fed80adc687a3eacfd5ee60fdcf9b5862d4b514d0884" gracePeriod=30
Oct 30 13:11:57.859084 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:57.859055    4943 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/ai-gateway-controller-697b76b867-4s264"
Oct 30 13:11:57.963694 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:57.963661    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/40dbf3f7-ccf2-4e8e-b5fb-7ea4e1fc42fd-metrics-certs\" (UniqueName: \"kubernetes.io/secret/40dbf3f7-ccf2-4e8e-b5fb-7ea4e1fc42fd-metrics-certs\") pod \"40dbf3f7-ccf2-4e8e-b5fb-7ea4e1fc42fd\" (UID: \"40dbf3f7-ccf2-4e8e-b5fb-7ea4e1fc42fd\") "
Oct 30 13:11:57.963990 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:57.963733    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/40dbf3f7-ccf2-4e8e-b5fb-7ea4e1fc42fd-kube-api-access-hjnqj\" (UniqueName: \"kubernetes.io/projected/40dbf3f7-ccf2-4e8e-b5fb-7ea4e1fc42fd-kube-api-access-hjnqj\") pod \"40dbf3f7-ccf2-4e8e-b5fb-7ea4e1fc42fd\" (UID: \"40dbf3f7-ccf2-4e8e-b5fb-7ea4e1fc42fd\") "
Oct 30 13:11:57.965955 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:57.965926    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/40dbf3f7-ccf2-4e8e-b5fb-7ea4e1fc42fd-metrics-certs" pod "40dbf3f7-ccf2-4e8e-b5fb-7ea4e1fc42fd" (UID: "40dbf3f7-ccf2-4e8e-b5fb-7ea4e1fc42fd"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 13:11:57.966095 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:57.965994    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/40dbf3f7-ccf2-4e8e-b5fb-7ea4e1fc42fd-kube-api-access-hjnqj" pod "40dbf3f7-ccf2-4e8e-b5fb-7ea4e1fc42fd" (UID: "40dbf3f7-ccf2-4e8e-b5fb-7ea4e1fc42fd"). InnerVolumeSpecName "kube-api-access-hjnqj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:11:58.011899 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:58.011821    4943 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-mlh2m"
Oct 30 13:11:58.012179 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:58.012152    4943 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-mlh2m"
Oct 30 13:11:58.013125 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:58.013104    4943 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-mlh2m"
Oct 30 13:11:58.065006 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:58.064980    4943 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/40dbf3f7-ccf2-4e8e-b5fb-7ea4e1fc42fd-metrics-certs\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:11:58.065006 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:58.065004    4943 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hjnqj\" (UniqueName: \"kubernetes.io/projected/40dbf3f7-ccf2-4e8e-b5fb-7ea4e1fc42fd-kube-api-access-hjnqj\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:11:58.547335 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:58.547296    4943 generic.go:356] "Generic (PLEG): container finished" podID="40dbf3f7-ccf2-4e8e-b5fb-7ea4e1fc42fd" containerID="d72afde53bf57fbe0aa7fed80adc687a3eacfd5ee60fdcf9b5862d4b514d0884" exitCode=0
Oct 30 13:11:58.547335 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:58.547337    4943 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/ai-gateway-controller-697b76b867-4s264"
Oct 30 13:11:58.547874 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:58.547375    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/ai-gateway-controller-697b76b867-4s264" event={"ID":"40dbf3f7-ccf2-4e8e-b5fb-7ea4e1fc42fd","Type":"ContainerDied","Data":"d72afde53bf57fbe0aa7fed80adc687a3eacfd5ee60fdcf9b5862d4b514d0884"}
Oct 30 13:11:58.547874 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:58.547426    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/ai-gateway-controller-697b76b867-4s264" event={"ID":"40dbf3f7-ccf2-4e8e-b5fb-7ea4e1fc42fd","Type":"ContainerDied","Data":"0a36ca8834121d52e2cb5c0e860abcfe4ce0f50de407c8b22639391ef67aa49f"}
Oct 30 13:11:58.547874 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:58.547449    4943 scope.go:122] "RemoveContainer" containerID="d72afde53bf57fbe0aa7fed80adc687a3eacfd5ee60fdcf9b5862d4b514d0884"
Oct 30 13:11:58.556378 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:58.556360    4943 scope.go:122] "RemoveContainer" containerID="d72afde53bf57fbe0aa7fed80adc687a3eacfd5ee60fdcf9b5862d4b514d0884"
Oct 30 13:11:58.556680 ip-10-0-142-84 kubenswrapper[4943]: E1030 13:11:58.556655    4943 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d72afde53bf57fbe0aa7fed80adc687a3eacfd5ee60fdcf9b5862d4b514d0884\": container with ID starting with d72afde53bf57fbe0aa7fed80adc687a3eacfd5ee60fdcf9b5862d4b514d0884 not found: ID does not exist" containerID="d72afde53bf57fbe0aa7fed80adc687a3eacfd5ee60fdcf9b5862d4b514d0884"
Oct 30 13:11:58.556732 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:58.556688    4943 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d72afde53bf57fbe0aa7fed80adc687a3eacfd5ee60fdcf9b5862d4b514d0884"} err="failed to get container status \"d72afde53bf57fbe0aa7fed80adc687a3eacfd5ee60fdcf9b5862d4b514d0884\": rpc error: code = NotFound desc = could not find container \"d72afde53bf57fbe0aa7fed80adc687a3eacfd5ee60fdcf9b5862d4b514d0884\": container with ID starting with d72afde53bf57fbe0aa7fed80adc687a3eacfd5ee60fdcf9b5862d4b514d0884 not found: ID does not exist"
Oct 30 13:11:58.605189 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:58.605161    4943 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/ai-gateway-controller-697b76b867-4s264"]
Oct 30 13:11:58.609350 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:11:58.609293    4943 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["opendatahub/ai-gateway-controller-697b76b867-4s264"]
Oct 30 13:12:00.349848 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:12:00.349816    4943 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="40dbf3f7-ccf2-4e8e-b5fb-7ea4e1fc42fd" path="/var/lib/kubelet/pods/40dbf3f7-ccf2-4e8e-b5fb-7ea4e1fc42fd/volumes"
Oct 30 13:12:04.429426 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:12:04.429398    4943 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ajkfrv_17f5a8d2-67e0-4489-8734-00f0d4483788/extract/0.log"
Oct 30 13:12:04.430624 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:12:04.430606    4943 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ajkfrv_17f5a8d2-67e0-4489-8734-00f0d4483788/extract/0.log"
Oct 30 13:12:04.441122 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:12:04.441103    4943 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-8fqfr_d8930ee7-0ae0-48b5-8ce3-cb8820355cef/console-operator/2.log"
Oct 30 13:12:04.442206 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:12:04.442189    4943 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-8fqfr_d8930ee7-0ae0-48b5-8ce3-cb8820355cef/console-operator/2.log"
Oct 30 13:12:05.954586 ip-10-0-142-84 kubenswrapper[4943]: E1030 13:12:05.954545    4943 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-knqz6" containerName="extractor"
Oct 30 13:12:35.955301 ip-10-0-142-84 kubenswrapper[4943]: E1030 13:12:35.955239    4943 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-knqz6" containerName="extractor"
Oct 30 13:13:05.505659 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:05.505625    4943 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-fdh7n"]
Oct 30 13:13:05.506135 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:05.506040    4943 cpu_manager.go:414] "containerMap: removing container" podUID="40dbf3f7-ccf2-4e8e-b5fb-7ea4e1fc42fd" containerName="manager"
Oct 30 13:13:05.506135 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:05.506051    4943 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="40dbf3f7-ccf2-4e8e-b5fb-7ea4e1fc42fd" containerName="manager"
Oct 30 13:13:05.506219 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:05.506142    4943 memory_manager.go:365] "RemoveStaleState removing state" podUID="40dbf3f7-ccf2-4e8e-b5fb-7ea4e1fc42fd" containerName="manager"
Oct 30 13:13:05.508595 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:05.508572    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-fdh7n"
Oct 30 13:13:05.527390 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:05.527370    4943 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w5-f8a55e-openshift-default-dockercfg-69hv5\""
Oct 30 13:13:05.541271 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:05.541224    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8c4e0447-d0df-4536-bea2-00f13e370c5b-istio-token\") pod \"e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-fdh7n\" (UID: \"8c4e0447-d0df-4536-bea2-00f13e370c5b\") " pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-fdh7n"
Oct 30 13:13:05.541380 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:05.541341    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-29rrj\" (UniqueName: \"kubernetes.io/projected/8c4e0447-d0df-4536-bea2-00f13e370c5b-kube-api-access-29rrj\") pod \"e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-fdh7n\" (UID: \"8c4e0447-d0df-4536-bea2-00f13e370c5b\") " pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-fdh7n"
Oct 30 13:13:05.541460 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:05.541397    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8c4e0447-d0df-4536-bea2-00f13e370c5b-istio-podinfo\") pod \"e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-fdh7n\" (UID: \"8c4e0447-d0df-4536-bea2-00f13e370c5b\") " pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-fdh7n"
Oct 30 13:13:05.541460 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:05.541426    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8c4e0447-d0df-4536-bea2-00f13e370c5b-credential-socket\") pod \"e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-fdh7n\" (UID: \"8c4e0447-d0df-4536-bea2-00f13e370c5b\") " pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-fdh7n"
Oct 30 13:13:05.541608 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:05.541585    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8c4e0447-d0df-4536-bea2-00f13e370c5b-istiod-ca-cert\") pod \"e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-fdh7n\" (UID: \"8c4e0447-d0df-4536-bea2-00f13e370c5b\") " pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-fdh7n"
Oct 30 13:13:05.541678 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:05.541635    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8c4e0447-d0df-4536-bea2-00f13e370c5b-workload-certs\") pod \"e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-fdh7n\" (UID: \"8c4e0447-d0df-4536-bea2-00f13e370c5b\") " pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-fdh7n"
Oct 30 13:13:05.541802 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:05.541676    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8c4e0447-d0df-4536-bea2-00f13e370c5b-workload-socket\") pod \"e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-fdh7n\" (UID: \"8c4e0447-d0df-4536-bea2-00f13e370c5b\") " pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-fdh7n"
Oct 30 13:13:05.541802 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:05.541744    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8c4e0447-d0df-4536-bea2-00f13e370c5b-istio-envoy\") pod \"e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-fdh7n\" (UID: \"8c4e0447-d0df-4536-bea2-00f13e370c5b\") " pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-fdh7n"
Oct 30 13:13:05.541937 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:05.541805    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8c4e0447-d0df-4536-bea2-00f13e370c5b-istio-data\") pod \"e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-fdh7n\" (UID: \"8c4e0447-d0df-4536-bea2-00f13e370c5b\") " pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-fdh7n"
Oct 30 13:13:05.544329 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:05.544295    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-fdh7n"]
Oct 30 13:13:05.642591 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:05.642553    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8c4e0447-d0df-4536-bea2-00f13e370c5b-istio-envoy\") pod \"e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-fdh7n\" (UID: \"8c4e0447-d0df-4536-bea2-00f13e370c5b\") " pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-fdh7n"
Oct 30 13:13:05.642591 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:05.642591    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8c4e0447-d0df-4536-bea2-00f13e370c5b-istio-data\") pod \"e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-fdh7n\" (UID: \"8c4e0447-d0df-4536-bea2-00f13e370c5b\") " pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-fdh7n"
Oct 30 13:13:05.642804 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:05.642630    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8c4e0447-d0df-4536-bea2-00f13e370c5b-istio-token\") pod \"e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-fdh7n\" (UID: \"8c4e0447-d0df-4536-bea2-00f13e370c5b\") " pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-fdh7n"
Oct 30 13:13:05.642804 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:05.642671    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-29rrj\" (UniqueName: \"kubernetes.io/projected/8c4e0447-d0df-4536-bea2-00f13e370c5b-kube-api-access-29rrj\") pod \"e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-fdh7n\" (UID: \"8c4e0447-d0df-4536-bea2-00f13e370c5b\") " pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-fdh7n"
Oct 30 13:13:05.642804 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:05.642719    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8c4e0447-d0df-4536-bea2-00f13e370c5b-istio-podinfo\") pod \"e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-fdh7n\" (UID: \"8c4e0447-d0df-4536-bea2-00f13e370c5b\") " pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-fdh7n"
Oct 30 13:13:05.642804 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:05.642764    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8c4e0447-d0df-4536-bea2-00f13e370c5b-credential-socket\") pod \"e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-fdh7n\" (UID: \"8c4e0447-d0df-4536-bea2-00f13e370c5b\") " pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-fdh7n"
Oct 30 13:13:05.642988 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:05.642833    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8c4e0447-d0df-4536-bea2-00f13e370c5b-istiod-ca-cert\") pod \"e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-fdh7n\" (UID: \"8c4e0447-d0df-4536-bea2-00f13e370c5b\") " pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-fdh7n"
Oct 30 13:13:05.642988 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:05.642862    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8c4e0447-d0df-4536-bea2-00f13e370c5b-workload-certs\") pod \"e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-fdh7n\" (UID: \"8c4e0447-d0df-4536-bea2-00f13e370c5b\") " pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-fdh7n"
Oct 30 13:13:05.642988 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:05.642898    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8c4e0447-d0df-4536-bea2-00f13e370c5b-workload-socket\") pod \"e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-fdh7n\" (UID: \"8c4e0447-d0df-4536-bea2-00f13e370c5b\") " pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-fdh7n"
Oct 30 13:13:05.643188 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:05.643111    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8c4e0447-d0df-4536-bea2-00f13e370c5b-istio-data\") pod \"e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-fdh7n\" (UID: \"8c4e0447-d0df-4536-bea2-00f13e370c5b\") " pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-fdh7n"
Oct 30 13:13:05.643246 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:05.643178    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8c4e0447-d0df-4536-bea2-00f13e370c5b-credential-socket\") pod \"e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-fdh7n\" (UID: \"8c4e0447-d0df-4536-bea2-00f13e370c5b\") " pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-fdh7n"
Oct 30 13:13:05.643336 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:05.643311    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8c4e0447-d0df-4536-bea2-00f13e370c5b-workload-certs\") pod \"e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-fdh7n\" (UID: \"8c4e0447-d0df-4536-bea2-00f13e370c5b\") " pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-fdh7n"
Oct 30 13:13:05.643588 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:05.643566    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8c4e0447-d0df-4536-bea2-00f13e370c5b-workload-socket\") pod \"e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-fdh7n\" (UID: \"8c4e0447-d0df-4536-bea2-00f13e370c5b\") " pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-fdh7n"
Oct 30 13:13:05.643651 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:05.643635    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8c4e0447-d0df-4536-bea2-00f13e370c5b-istiod-ca-cert\") pod \"e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-fdh7n\" (UID: \"8c4e0447-d0df-4536-bea2-00f13e370c5b\") " pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-fdh7n"
Oct 30 13:13:05.645016 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:05.644987    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8c4e0447-d0df-4536-bea2-00f13e370c5b-istio-envoy\") pod \"e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-fdh7n\" (UID: \"8c4e0447-d0df-4536-bea2-00f13e370c5b\") " pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-fdh7n"
Oct 30 13:13:05.645142 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:05.645127    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8c4e0447-d0df-4536-bea2-00f13e370c5b-istio-podinfo\") pod \"e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-fdh7n\" (UID: \"8c4e0447-d0df-4536-bea2-00f13e370c5b\") " pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-fdh7n"
Oct 30 13:13:05.670564 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:05.670540    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8c4e0447-d0df-4536-bea2-00f13e370c5b-istio-token\") pod \"e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-fdh7n\" (UID: \"8c4e0447-d0df-4536-bea2-00f13e370c5b\") " pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-fdh7n"
Oct 30 13:13:05.679803 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:05.679779    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-29rrj\" (UniqueName: \"kubernetes.io/projected/8c4e0447-d0df-4536-bea2-00f13e370c5b-kube-api-access-29rrj\") pod \"e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-fdh7n\" (UID: \"8c4e0447-d0df-4536-bea2-00f13e370c5b\") " pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-fdh7n"
Oct 30 13:13:05.680038 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:05.680016    4943 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-l4l9h"]
Oct 30 13:13:05.683708 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:05.683691    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-l4l9h"
Oct 30 13:13:05.710287 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:05.710256    4943 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w6-ab3176-openshift-default-dockercfg-ws7g2\""
Oct 30 13:13:05.718897 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:05.718875    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-l4l9h"]
Oct 30 13:13:05.743482 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:05.743458    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e9509589-96c1-4883-a552-86fd112c147f-istiod-ca-cert\") pod \"e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-l4l9h\" (UID: \"e9509589-96c1-4883-a552-86fd112c147f\") " pod="openshift-ingress/e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-l4l9h"
Oct 30 13:13:05.743600 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:05.743503    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e9509589-96c1-4883-a552-86fd112c147f-istio-podinfo\") pod \"e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-l4l9h\" (UID: \"e9509589-96c1-4883-a552-86fd112c147f\") " pod="openshift-ingress/e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-l4l9h"
Oct 30 13:13:05.743650 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:05.743627    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e9509589-96c1-4883-a552-86fd112c147f-istio-data\") pod \"e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-l4l9h\" (UID: \"e9509589-96c1-4883-a552-86fd112c147f\") " pod="openshift-ingress/e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-l4l9h"
Oct 30 13:13:05.743696 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:05.743665    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2c9j4\" (UniqueName: \"kubernetes.io/projected/e9509589-96c1-4883-a552-86fd112c147f-kube-api-access-2c9j4\") pod \"e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-l4l9h\" (UID: \"e9509589-96c1-4883-a552-86fd112c147f\") " pod="openshift-ingress/e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-l4l9h"
Oct 30 13:13:05.743850 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:05.743826    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e9509589-96c1-4883-a552-86fd112c147f-workload-certs\") pod \"e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-l4l9h\" (UID: \"e9509589-96c1-4883-a552-86fd112c147f\") " pod="openshift-ingress/e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-l4l9h"
Oct 30 13:13:05.743898 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:05.743880    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e9509589-96c1-4883-a552-86fd112c147f-istio-token\") pod \"e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-l4l9h\" (UID: \"e9509589-96c1-4883-a552-86fd112c147f\") " pod="openshift-ingress/e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-l4l9h"
Oct 30 13:13:05.743939 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:05.743916    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e9509589-96c1-4883-a552-86fd112c147f-credential-socket\") pod \"e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-l4l9h\" (UID: \"e9509589-96c1-4883-a552-86fd112c147f\") " pod="openshift-ingress/e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-l4l9h"
Oct 30 13:13:05.743986 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:05.743972    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e9509589-96c1-4883-a552-86fd112c147f-workload-socket\") pod \"e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-l4l9h\" (UID: \"e9509589-96c1-4883-a552-86fd112c147f\") " pod="openshift-ingress/e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-l4l9h"
Oct 30 13:13:05.744023 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:05.744004    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e9509589-96c1-4883-a552-86fd112c147f-istio-envoy\") pod \"e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-l4l9h\" (UID: \"e9509589-96c1-4883-a552-86fd112c147f\") " pod="openshift-ingress/e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-l4l9h"
Oct 30 13:13:05.821151 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:05.821086    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-fdh7n"
Oct 30 13:13:05.844558 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:05.844530    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e9509589-96c1-4883-a552-86fd112c147f-istio-token\") pod \"e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-l4l9h\" (UID: \"e9509589-96c1-4883-a552-86fd112c147f\") " pod="openshift-ingress/e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-l4l9h"
Oct 30 13:13:05.844694 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:05.844577    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e9509589-96c1-4883-a552-86fd112c147f-credential-socket\") pod \"e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-l4l9h\" (UID: \"e9509589-96c1-4883-a552-86fd112c147f\") " pod="openshift-ingress/e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-l4l9h"
Oct 30 13:13:05.844763 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:05.844698    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e9509589-96c1-4883-a552-86fd112c147f-workload-socket\") pod \"e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-l4l9h\" (UID: \"e9509589-96c1-4883-a552-86fd112c147f\") " pod="openshift-ingress/e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-l4l9h"
Oct 30 13:13:05.844828 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:05.844782    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e9509589-96c1-4883-a552-86fd112c147f-istio-envoy\") pod \"e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-l4l9h\" (UID: \"e9509589-96c1-4883-a552-86fd112c147f\") " pod="openshift-ingress/e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-l4l9h"
Oct 30 13:13:05.845122 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:05.844845    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e9509589-96c1-4883-a552-86fd112c147f-istiod-ca-cert\") pod \"e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-l4l9h\" (UID: \"e9509589-96c1-4883-a552-86fd112c147f\") " pod="openshift-ingress/e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-l4l9h"
Oct 30 13:13:05.845122 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:05.844892    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e9509589-96c1-4883-a552-86fd112c147f-istio-podinfo\") pod \"e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-l4l9h\" (UID: \"e9509589-96c1-4883-a552-86fd112c147f\") " pod="openshift-ingress/e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-l4l9h"
Oct 30 13:13:05.845122 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:05.844971    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e9509589-96c1-4883-a552-86fd112c147f-credential-socket\") pod \"e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-l4l9h\" (UID: \"e9509589-96c1-4883-a552-86fd112c147f\") " pod="openshift-ingress/e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-l4l9h"
Oct 30 13:13:05.845122 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:05.844987    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e9509589-96c1-4883-a552-86fd112c147f-istio-data\") pod \"e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-l4l9h\" (UID: \"e9509589-96c1-4883-a552-86fd112c147f\") " pod="openshift-ingress/e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-l4l9h"
Oct 30 13:13:05.845533 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:05.845477    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2c9j4\" (UniqueName: \"kubernetes.io/projected/e9509589-96c1-4883-a552-86fd112c147f-kube-api-access-2c9j4\") pod \"e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-l4l9h\" (UID: \"e9509589-96c1-4883-a552-86fd112c147f\") " pod="openshift-ingress/e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-l4l9h"
Oct 30 13:13:05.845757 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:05.845720    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e9509589-96c1-4883-a552-86fd112c147f-istio-data\") pod \"e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-l4l9h\" (UID: \"e9509589-96c1-4883-a552-86fd112c147f\") " pod="openshift-ingress/e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-l4l9h"
Oct 30 13:13:05.845869 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:05.845774    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e9509589-96c1-4883-a552-86fd112c147f-workload-certs\") pod \"e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-l4l9h\" (UID: \"e9509589-96c1-4883-a552-86fd112c147f\") " pod="openshift-ingress/e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-l4l9h"
Oct 30 13:13:05.846039 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:05.846019    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e9509589-96c1-4883-a552-86fd112c147f-workload-socket\") pod \"e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-l4l9h\" (UID: \"e9509589-96c1-4883-a552-86fd112c147f\") " pod="openshift-ingress/e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-l4l9h"
Oct 30 13:13:05.846345 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:05.846319    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e9509589-96c1-4883-a552-86fd112c147f-workload-certs\") pod \"e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-l4l9h\" (UID: \"e9509589-96c1-4883-a552-86fd112c147f\") " pod="openshift-ingress/e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-l4l9h"
Oct 30 13:13:05.846688 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:05.846658    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e9509589-96c1-4883-a552-86fd112c147f-istiod-ca-cert\") pod \"e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-l4l9h\" (UID: \"e9509589-96c1-4883-a552-86fd112c147f\") " pod="openshift-ingress/e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-l4l9h"
Oct 30 13:13:05.847889 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:05.847864    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e9509589-96c1-4883-a552-86fd112c147f-istio-podinfo\") pod \"e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-l4l9h\" (UID: \"e9509589-96c1-4883-a552-86fd112c147f\") " pod="openshift-ingress/e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-l4l9h"
Oct 30 13:13:05.848837 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:05.848806    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e9509589-96c1-4883-a552-86fd112c147f-istio-envoy\") pod \"e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-l4l9h\" (UID: \"e9509589-96c1-4883-a552-86fd112c147f\") " pod="openshift-ingress/e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-l4l9h"
Oct 30 13:13:05.873062 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:05.873012    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e9509589-96c1-4883-a552-86fd112c147f-istio-token\") pod \"e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-l4l9h\" (UID: \"e9509589-96c1-4883-a552-86fd112c147f\") " pod="openshift-ingress/e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-l4l9h"
Oct 30 13:13:05.873760 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:05.873741    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2c9j4\" (UniqueName: \"kubernetes.io/projected/e9509589-96c1-4883-a552-86fd112c147f-kube-api-access-2c9j4\") pod \"e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-l4l9h\" (UID: \"e9509589-96c1-4883-a552-86fd112c147f\") " pod="openshift-ingress/e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-l4l9h"
Oct 30 13:13:05.956591 ip-10-0-142-84 kubenswrapper[4943]: E1030 13:13:05.956553    4943 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-knqz6" containerName="extractor"
Oct 30 13:13:06.001424 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:06.001392    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-l4l9h"
Oct 30 13:13:06.150971 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:06.150512    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-fdh7n"]
Oct 30 13:13:06.155929 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:06.153783    4943 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-qzz29"]
Oct 30 13:13:06.155929 ip-10-0-142-84 kubenswrapper[4943]: W1030 13:13:06.155058    4943 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8c4e0447_d0df_4536_bea2_00f13e370c5b.slice/crio-89d4728d41eda972cca81436a97795f4e831de63c9981162f9051d6ef8533076 WatchSource:0}: Error finding container 89d4728d41eda972cca81436a97795f4e831de63c9981162f9051d6ef8533076: Status 404 returned error can't find the container with id 89d4728d41eda972cca81436a97795f4e831de63c9981162f9051d6ef8533076
Oct 30 13:13:06.156675 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:06.156645    4943 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 13:13:06.156782 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:06.156700    4943 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 13:13:06.156782 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:06.156737    4943 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 13:13:06.157966 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:06.157945    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-qzz29"
Oct 30 13:13:06.187772 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:06.187744    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-qzz29"]
Oct 30 13:13:06.230224 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:06.230188    4943 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w4-88f6a4-openshift-default-dockercfg-tgcfz\""
Oct 30 13:13:06.249120 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:06.249094    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7bae0af9-fbeb-4480-9452-1abab86550e8-workload-certs\") pod \"e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-qzz29\" (UID: \"7bae0af9-fbeb-4480-9452-1abab86550e8\") " pod="openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-qzz29"
Oct 30 13:13:06.249247 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:06.249127    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7bae0af9-fbeb-4480-9452-1abab86550e8-istio-podinfo\") pod \"e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-qzz29\" (UID: \"7bae0af9-fbeb-4480-9452-1abab86550e8\") " pod="openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-qzz29"
Oct 30 13:13:06.249247 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:06.249155    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7bae0af9-fbeb-4480-9452-1abab86550e8-istio-envoy\") pod \"e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-qzz29\" (UID: \"7bae0af9-fbeb-4480-9452-1abab86550e8\") " pod="openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-qzz29"
Oct 30 13:13:06.249436 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:06.249401    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7bae0af9-fbeb-4480-9452-1abab86550e8-istio-data\") pod \"e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-qzz29\" (UID: \"7bae0af9-fbeb-4480-9452-1abab86550e8\") " pod="openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-qzz29"
Oct 30 13:13:06.249577 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:06.249560    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7bae0af9-fbeb-4480-9452-1abab86550e8-istiod-ca-cert\") pod \"e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-qzz29\" (UID: \"7bae0af9-fbeb-4480-9452-1abab86550e8\") " pod="openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-qzz29"
Oct 30 13:13:06.249635 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:06.249613    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7bae0af9-fbeb-4480-9452-1abab86550e8-workload-socket\") pod \"e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-qzz29\" (UID: \"7bae0af9-fbeb-4480-9452-1abab86550e8\") " pod="openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-qzz29"
Oct 30 13:13:06.249681 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:06.249656    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jdh6c\" (UniqueName: \"kubernetes.io/projected/7bae0af9-fbeb-4480-9452-1abab86550e8-kube-api-access-jdh6c\") pod \"e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-qzz29\" (UID: \"7bae0af9-fbeb-4480-9452-1abab86550e8\") " pod="openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-qzz29"
Oct 30 13:13:06.249717 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:06.249680    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7bae0af9-fbeb-4480-9452-1abab86550e8-credential-socket\") pod \"e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-qzz29\" (UID: \"7bae0af9-fbeb-4480-9452-1abab86550e8\") " pod="openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-qzz29"
Oct 30 13:13:06.249800 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:06.249782    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7bae0af9-fbeb-4480-9452-1abab86550e8-istio-token\") pod \"e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-qzz29\" (UID: \"7bae0af9-fbeb-4480-9452-1abab86550e8\") " pod="openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-qzz29"
Oct 30 13:13:06.358844 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:06.351693    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jdh6c\" (UniqueName: \"kubernetes.io/projected/7bae0af9-fbeb-4480-9452-1abab86550e8-kube-api-access-jdh6c\") pod \"e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-qzz29\" (UID: \"7bae0af9-fbeb-4480-9452-1abab86550e8\") " pod="openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-qzz29"
Oct 30 13:13:06.358844 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:06.351754    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7bae0af9-fbeb-4480-9452-1abab86550e8-credential-socket\") pod \"e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-qzz29\" (UID: \"7bae0af9-fbeb-4480-9452-1abab86550e8\") " pod="openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-qzz29"
Oct 30 13:13:06.358844 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:06.351826    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7bae0af9-fbeb-4480-9452-1abab86550e8-istio-token\") pod \"e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-qzz29\" (UID: \"7bae0af9-fbeb-4480-9452-1abab86550e8\") " pod="openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-qzz29"
Oct 30 13:13:06.358844 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:06.351882    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7bae0af9-fbeb-4480-9452-1abab86550e8-workload-certs\") pod \"e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-qzz29\" (UID: \"7bae0af9-fbeb-4480-9452-1abab86550e8\") " pod="openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-qzz29"
Oct 30 13:13:06.358844 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:06.351914    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7bae0af9-fbeb-4480-9452-1abab86550e8-istio-podinfo\") pod \"e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-qzz29\" (UID: \"7bae0af9-fbeb-4480-9452-1abab86550e8\") " pod="openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-qzz29"
Oct 30 13:13:06.358844 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:06.351955    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7bae0af9-fbeb-4480-9452-1abab86550e8-istio-envoy\") pod \"e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-qzz29\" (UID: \"7bae0af9-fbeb-4480-9452-1abab86550e8\") " pod="openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-qzz29"
Oct 30 13:13:06.358844 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:06.351992    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7bae0af9-fbeb-4480-9452-1abab86550e8-istio-data\") pod \"e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-qzz29\" (UID: \"7bae0af9-fbeb-4480-9452-1abab86550e8\") " pod="openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-qzz29"
Oct 30 13:13:06.358844 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:06.352078    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7bae0af9-fbeb-4480-9452-1abab86550e8-istiod-ca-cert\") pod \"e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-qzz29\" (UID: \"7bae0af9-fbeb-4480-9452-1abab86550e8\") " pod="openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-qzz29"
Oct 30 13:13:06.358844 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:06.352118    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7bae0af9-fbeb-4480-9452-1abab86550e8-workload-socket\") pod \"e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-qzz29\" (UID: \"7bae0af9-fbeb-4480-9452-1abab86550e8\") " pod="openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-qzz29"
Oct 30 13:13:06.358844 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:06.352738    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7bae0af9-fbeb-4480-9452-1abab86550e8-workload-socket\") pod \"e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-qzz29\" (UID: \"7bae0af9-fbeb-4480-9452-1abab86550e8\") " pod="openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-qzz29"
Oct 30 13:13:06.358844 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:06.352912    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7bae0af9-fbeb-4480-9452-1abab86550e8-workload-certs\") pod \"e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-qzz29\" (UID: \"7bae0af9-fbeb-4480-9452-1abab86550e8\") " pod="openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-qzz29"
Oct 30 13:13:06.358844 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:06.353734    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7bae0af9-fbeb-4480-9452-1abab86550e8-credential-socket\") pod \"e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-qzz29\" (UID: \"7bae0af9-fbeb-4480-9452-1abab86550e8\") " pod="openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-qzz29"
Oct 30 13:13:06.358844 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:06.353896    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7bae0af9-fbeb-4480-9452-1abab86550e8-istio-data\") pod \"e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-qzz29\" (UID: \"7bae0af9-fbeb-4480-9452-1abab86550e8\") " pod="openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-qzz29"
Oct 30 13:13:06.358844 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:06.354709    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7bae0af9-fbeb-4480-9452-1abab86550e8-istiod-ca-cert\") pod \"e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-qzz29\" (UID: \"7bae0af9-fbeb-4480-9452-1abab86550e8\") " pod="openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-qzz29"
Oct 30 13:13:06.358844 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:06.356246    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7bae0af9-fbeb-4480-9452-1abab86550e8-istio-podinfo\") pod \"e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-qzz29\" (UID: \"7bae0af9-fbeb-4480-9452-1abab86550e8\") " pod="openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-qzz29"
Oct 30 13:13:06.362885 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:06.360004    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7bae0af9-fbeb-4480-9452-1abab86550e8-istio-envoy\") pod \"e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-qzz29\" (UID: \"7bae0af9-fbeb-4480-9452-1abab86550e8\") " pod="openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-qzz29"
Oct 30 13:13:06.362885 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:06.360666    4943 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 13:13:06.362885 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:06.360793    4943 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 13:13:06.362885 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:06.360901    4943 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 13:13:06.366479 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:06.366457    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-l4l9h"]
Oct 30 13:13:06.378891 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:06.378869    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jdh6c\" (UniqueName: \"kubernetes.io/projected/7bae0af9-fbeb-4480-9452-1abab86550e8-kube-api-access-jdh6c\") pod \"e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-qzz29\" (UID: \"7bae0af9-fbeb-4480-9452-1abab86550e8\") " pod="openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-qzz29"
Oct 30 13:13:06.379966 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:06.379842    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7bae0af9-fbeb-4480-9452-1abab86550e8-istio-token\") pod \"e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-qzz29\" (UID: \"7bae0af9-fbeb-4480-9452-1abab86550e8\") " pod="openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-qzz29"
Oct 30 13:13:06.474612 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:06.474576    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-qzz29"
Oct 30 13:13:06.815213 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:06.815184    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-qzz29"]
Oct 30 13:13:06.817219 ip-10-0-142-84 kubenswrapper[4943]: W1030 13:13:06.817177    4943 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7bae0af9_fbeb_4480_9452_1abab86550e8.slice/crio-b318e29accd359fd5d6ffa4c003837d9647691d64c64b464f10c3711b6bfc94d WatchSource:0}: Error finding container b318e29accd359fd5d6ffa4c003837d9647691d64c64b464f10c3711b6bfc94d: Status 404 returned error can't find the container with id b318e29accd359fd5d6ffa4c003837d9647691d64c64b464f10c3711b6bfc94d
Oct 30 13:13:06.819823 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:06.819788    4943 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 13:13:06.819906 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:06.819858    4943 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 13:13:06.819906 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:06.819890    4943 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 13:13:06.911409 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:06.911369    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-l4l9h" event={"ID":"e9509589-96c1-4883-a552-86fd112c147f","Type":"ContainerStarted","Data":"8ba842ec132925a085d6d2ecbee87fb111bd4c7e6c15faada2102de24f212d34"}
Oct 30 13:13:06.911578 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:06.911439    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-l4l9h" event={"ID":"e9509589-96c1-4883-a552-86fd112c147f","Type":"ContainerStarted","Data":"5ca1a5fa65a9e91ded7acf02a70eca156a7d250dc3286bc3ac05ca5d5130c262"}
Oct 30 13:13:06.914356 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:06.914324    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-fdh7n" event={"ID":"8c4e0447-d0df-4536-bea2-00f13e370c5b","Type":"ContainerStarted","Data":"f8327c30b96218ee8c871b6cded44493a4bbe2ef4d8aeacae77de64292f0f756"}
Oct 30 13:13:06.914550 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:06.914383    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-fdh7n" event={"ID":"8c4e0447-d0df-4536-bea2-00f13e370c5b","Type":"ContainerStarted","Data":"89d4728d41eda972cca81436a97795f4e831de63c9981162f9051d6ef8533076"}
Oct 30 13:13:06.916103 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:06.916062    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-qzz29" event={"ID":"7bae0af9-fbeb-4480-9452-1abab86550e8","Type":"ContainerStarted","Data":"c36000632379947fbb591c5b3f97b05877896b6ee819563caf1193313eaab073"}
Oct 30 13:13:06.916236 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:06.916105    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-qzz29" event={"ID":"7bae0af9-fbeb-4480-9452-1abab86550e8","Type":"ContainerStarted","Data":"b318e29accd359fd5d6ffa4c003837d9647691d64c64b464f10c3711b6bfc94d"}
Oct 30 13:13:06.966458 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:06.966393    4943 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-l4l9h" podStartSLOduration=1.966375006 podStartE2EDuration="1.966375006s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:13:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:13:06.961641848 +0000 UTC m=+962.749645591" watchObservedRunningTime="2026-10-30 13:13:06.966375006 +0000 UTC m=+962.754378734"
Oct 30 13:13:07.001590 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:07.001556    4943 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-l4l9h"
Oct 30 13:13:07.006900 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:07.006871    4943 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-l4l9h"
Oct 30 13:13:07.022712 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:07.022651    4943 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-fdh7n" podStartSLOduration=2.022635617 podStartE2EDuration="2.022635617s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:13:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:13:07.018670206 +0000 UTC m=+962.806673938" watchObservedRunningTime="2026-10-30 13:13:07.022635617 +0000 UTC m=+962.810639348"
Oct 30 13:13:07.075873 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:07.075774    4943 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-qzz29" podStartSLOduration=1.075755995 podStartE2EDuration="1.075755995s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:13:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:13:07.072474023 +0000 UTC m=+962.860477754" watchObservedRunningTime="2026-10-30 13:13:07.075755995 +0000 UTC m=+962.863759727"
Oct 30 13:13:07.474918 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:07.474882    4943 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-qzz29"
Oct 30 13:13:07.480566 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:07.480540    4943 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-qzz29"
Oct 30 13:13:07.821348 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:07.821242    4943 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-fdh7n"
Oct 30 13:13:07.826454 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:07.826426    4943 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-fdh7n"
Oct 30 13:13:07.922253 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:07.922222    4943 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-fdh7n"
Oct 30 13:13:07.922556 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:07.922528    4943 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-l4l9h"
Oct 30 13:13:07.922698 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:07.922571    4943 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-qzz29"
Oct 30 13:13:07.922985 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:07.922963    4943 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-fdh7n"
Oct 30 13:13:07.923113 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:07.923090    4943 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-l4l9h"
Oct 30 13:13:07.923205 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:07.923158    4943 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-qzz29"
Oct 30 13:13:11.489043 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:11.489007    4943 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-031737-57d4df6b68-xr5s2"]
Oct 30 13:13:11.492215 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:11.492188    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w0-031737-57d4df6b68-xr5s2"
Oct 30 13:13:11.533285 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:11.525458    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-031737-57d4df6b68-xr5s2"]
Oct 30 13:13:11.582315 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:11.581731    4943 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w0-031737\""
Oct 30 13:13:11.589496 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:11.589462    4943 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w0-031737-dockercfg-bh9nd\""
Oct 30 13:13:11.605935 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:11.605902    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9fc1504d-012b-4afd-bebf-7ac106d6990e-plugins-config-volume\") pod \"payload-processing-e2e-worker-w0-031737-57d4df6b68-xr5s2\" (UID: \"9fc1504d-012b-4afd-bebf-7ac106d6990e\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-031737-57d4df6b68-xr5s2"
Oct 30 13:13:11.606079 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:11.605942    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j79vn\" (UniqueName: \"kubernetes.io/projected/9fc1504d-012b-4afd-bebf-7ac106d6990e-kube-api-access-j79vn\") pod \"payload-processing-e2e-worker-w0-031737-57d4df6b68-xr5s2\" (UID: \"9fc1504d-012b-4afd-bebf-7ac106d6990e\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-031737-57d4df6b68-xr5s2"
Oct 30 13:13:11.707072 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:11.707037    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9fc1504d-012b-4afd-bebf-7ac106d6990e-plugins-config-volume\") pod \"payload-processing-e2e-worker-w0-031737-57d4df6b68-xr5s2\" (UID: \"9fc1504d-012b-4afd-bebf-7ac106d6990e\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-031737-57d4df6b68-xr5s2"
Oct 30 13:13:11.707229 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:11.707076    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j79vn\" (UniqueName: \"kubernetes.io/projected/9fc1504d-012b-4afd-bebf-7ac106d6990e-kube-api-access-j79vn\") pod \"payload-processing-e2e-worker-w0-031737-57d4df6b68-xr5s2\" (UID: \"9fc1504d-012b-4afd-bebf-7ac106d6990e\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-031737-57d4df6b68-xr5s2"
Oct 30 13:13:11.707839 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:11.707820    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9fc1504d-012b-4afd-bebf-7ac106d6990e-plugins-config-volume\") pod \"payload-processing-e2e-worker-w0-031737-57d4df6b68-xr5s2\" (UID: \"9fc1504d-012b-4afd-bebf-7ac106d6990e\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-031737-57d4df6b68-xr5s2"
Oct 30 13:13:11.737764 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:11.737731    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j79vn\" (UniqueName: \"kubernetes.io/projected/9fc1504d-012b-4afd-bebf-7ac106d6990e-kube-api-access-j79vn\") pod \"payload-processing-e2e-worker-w0-031737-57d4df6b68-xr5s2\" (UID: \"9fc1504d-012b-4afd-bebf-7ac106d6990e\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-031737-57d4df6b68-xr5s2"
Oct 30 13:13:11.805881 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:11.805805    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w0-031737-57d4df6b68-xr5s2"
Oct 30 13:13:12.111079 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:12.111042    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-031737-57d4df6b68-xr5s2"]
Oct 30 13:13:12.111886 ip-10-0-142-84 kubenswrapper[4943]: W1030 13:13:12.111856    4943 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9fc1504d_012b_4afd_bebf_7ac106d6990e.slice/crio-44db439d26890bb6e3ee341c77ccc77aa448ca1aa0175b9e09265a1cc2694701 WatchSource:0}: Error finding container 44db439d26890bb6e3ee341c77ccc77aa448ca1aa0175b9e09265a1cc2694701: Status 404 returned error can't find the container with id 44db439d26890bb6e3ee341c77ccc77aa448ca1aa0175b9e09265a1cc2694701
Oct 30 13:13:12.959902 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:12.959860    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w0-031737-57d4df6b68-xr5s2" event={"ID":"9fc1504d-012b-4afd-bebf-7ac106d6990e","Type":"ContainerStarted","Data":"2f655124e975981258127d10703872d21c94f5f1dedd7ae7c29ca637f5fbbd4b"}
Oct 30 13:13:12.960322 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:12.959922    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w0-031737-57d4df6b68-xr5s2" event={"ID":"9fc1504d-012b-4afd-bebf-7ac106d6990e","Type":"ContainerStarted","Data":"44db439d26890bb6e3ee341c77ccc77aa448ca1aa0175b9e09265a1cc2694701"}
Oct 30 13:13:12.960322 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:12.959965    4943 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-w0-031737-57d4df6b68-xr5s2"
Oct 30 13:13:13.015060 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:13.015000    4943 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-w0-031737-57d4df6b68-xr5s2" podStartSLOduration=2.014984051 podStartE2EDuration="2.014984051s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:13:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:13:13.012738488 +0000 UTC m=+968.800742217" watchObservedRunningTime="2026-10-30 13:13:13.014984051 +0000 UTC m=+968.802987780"
Oct 30 13:13:13.495785 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:13.486490    4943 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-88f6a4-7dd558b944-gkzck"]
Oct 30 13:13:13.495785 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:13.490347    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w4-88f6a4-7dd558b944-gkzck"
Oct 30 13:13:13.499043 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:13.499002    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-88f6a4-7dd558b944-gkzck"]
Oct 30 13:13:13.509577 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:13.509547    4943 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w4-88f6a4\""
Oct 30 13:13:13.509782 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:13.509683    4943 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w4-88f6a4-dockercfg-7fhkf\""
Oct 30 13:13:13.573903 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:13.573870    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l7t5v\" (UniqueName: \"kubernetes.io/projected/80c84ecd-0a11-4d8e-ab46-2ba9efd4e676-kube-api-access-l7t5v\") pod \"payload-processing-e2e-worker-w4-88f6a4-7dd558b944-gkzck\" (UID: \"80c84ecd-0a11-4d8e-ab46-2ba9efd4e676\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-88f6a4-7dd558b944-gkzck"
Oct 30 13:13:13.574026 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:13.573978    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/80c84ecd-0a11-4d8e-ab46-2ba9efd4e676-plugins-config-volume\") pod \"payload-processing-e2e-worker-w4-88f6a4-7dd558b944-gkzck\" (UID: \"80c84ecd-0a11-4d8e-ab46-2ba9efd4e676\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-88f6a4-7dd558b944-gkzck"
Oct 30 13:13:13.675223 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:13.675187    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l7t5v\" (UniqueName: \"kubernetes.io/projected/80c84ecd-0a11-4d8e-ab46-2ba9efd4e676-kube-api-access-l7t5v\") pod \"payload-processing-e2e-worker-w4-88f6a4-7dd558b944-gkzck\" (UID: \"80c84ecd-0a11-4d8e-ab46-2ba9efd4e676\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-88f6a4-7dd558b944-gkzck"
Oct 30 13:13:13.675385 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:13.675287    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/80c84ecd-0a11-4d8e-ab46-2ba9efd4e676-plugins-config-volume\") pod \"payload-processing-e2e-worker-w4-88f6a4-7dd558b944-gkzck\" (UID: \"80c84ecd-0a11-4d8e-ab46-2ba9efd4e676\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-88f6a4-7dd558b944-gkzck"
Oct 30 13:13:13.676000 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:13.675976    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/80c84ecd-0a11-4d8e-ab46-2ba9efd4e676-plugins-config-volume\") pod \"payload-processing-e2e-worker-w4-88f6a4-7dd558b944-gkzck\" (UID: \"80c84ecd-0a11-4d8e-ab46-2ba9efd4e676\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-88f6a4-7dd558b944-gkzck"
Oct 30 13:13:13.700946 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:13.700916    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l7t5v\" (UniqueName: \"kubernetes.io/projected/80c84ecd-0a11-4d8e-ab46-2ba9efd4e676-kube-api-access-l7t5v\") pod \"payload-processing-e2e-worker-w4-88f6a4-7dd558b944-gkzck\" (UID: \"80c84ecd-0a11-4d8e-ab46-2ba9efd4e676\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-88f6a4-7dd558b944-gkzck"
Oct 30 13:13:13.813960 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:13.813887    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w4-88f6a4-7dd558b944-gkzck"
Oct 30 13:13:14.119888 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:14.119860    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-88f6a4-7dd558b944-gkzck"]
Oct 30 13:13:14.120836 ip-10-0-142-84 kubenswrapper[4943]: W1030 13:13:14.120805    4943 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod80c84ecd_0a11_4d8e_ab46_2ba9efd4e676.slice/crio-26347a503a412a0e520ba88ee140cc39344c1f9049422bd5439af101eeabcbe3 WatchSource:0}: Error finding container 26347a503a412a0e520ba88ee140cc39344c1f9049422bd5439af101eeabcbe3: Status 404 returned error can't find the container with id 26347a503a412a0e520ba88ee140cc39344c1f9049422bd5439af101eeabcbe3
Oct 30 13:13:14.352580 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:14.352551    4943 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w3-d0a18c-778df4584c-4f9hx"]
Oct 30 13:13:14.355407 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:14.355388    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-d0a18c-778df4584c-4f9hx"
Oct 30 13:13:14.375527 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:14.375461    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w3-d0a18c-778df4584c-4f9hx"]
Oct 30 13:13:14.376130 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:14.376108    4943 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w3-d0a18c\""
Oct 30 13:13:14.387368 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:14.387347    4943 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w3-d0a18c-dockercfg-rxr9x\""
Oct 30 13:13:14.484768 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:14.484736    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/93220213-580a-465b-b060-3cea89bfd247-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w3-d0a18c-778df4584c-4f9hx\" (UID: \"93220213-580a-465b-b060-3cea89bfd247\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-d0a18c-778df4584c-4f9hx"
Oct 30 13:13:14.484920 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:14.484873    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n847r\" (UniqueName: \"kubernetes.io/projected/93220213-580a-465b-b060-3cea89bfd247-kube-api-access-n847r\") pod \"payload-pre-processing-e2e-worker-w3-d0a18c-778df4584c-4f9hx\" (UID: \"93220213-580a-465b-b060-3cea89bfd247\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-d0a18c-778df4584c-4f9hx"
Oct 30 13:13:14.585919 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:14.585882    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n847r\" (UniqueName: \"kubernetes.io/projected/93220213-580a-465b-b060-3cea89bfd247-kube-api-access-n847r\") pod \"payload-pre-processing-e2e-worker-w3-d0a18c-778df4584c-4f9hx\" (UID: \"93220213-580a-465b-b060-3cea89bfd247\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-d0a18c-778df4584c-4f9hx"
Oct 30 13:13:14.586118 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:14.585984    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/93220213-580a-465b-b060-3cea89bfd247-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w3-d0a18c-778df4584c-4f9hx\" (UID: \"93220213-580a-465b-b060-3cea89bfd247\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-d0a18c-778df4584c-4f9hx"
Oct 30 13:13:14.586774 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:14.586744    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/93220213-580a-465b-b060-3cea89bfd247-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w3-d0a18c-778df4584c-4f9hx\" (UID: \"93220213-580a-465b-b060-3cea89bfd247\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-d0a18c-778df4584c-4f9hx"
Oct 30 13:13:14.612363 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:14.612338    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n847r\" (UniqueName: \"kubernetes.io/projected/93220213-580a-465b-b060-3cea89bfd247-kube-api-access-n847r\") pod \"payload-pre-processing-e2e-worker-w3-d0a18c-778df4584c-4f9hx\" (UID: \"93220213-580a-465b-b060-3cea89bfd247\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-d0a18c-778df4584c-4f9hx"
Oct 30 13:13:14.667070 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:14.667006    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-d0a18c-778df4584c-4f9hx"
Oct 30 13:13:14.972744 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:14.972709    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w4-88f6a4-7dd558b944-gkzck" event={"ID":"80c84ecd-0a11-4d8e-ab46-2ba9efd4e676","Type":"ContainerStarted","Data":"bd85b10273b50dd2fca89ae5428adcff4261b0824c739153bd1148b75a648e9f"}
Oct 30 13:13:14.972911 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:14.972795    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w4-88f6a4-7dd558b944-gkzck" event={"ID":"80c84ecd-0a11-4d8e-ab46-2ba9efd4e676","Type":"ContainerStarted","Data":"26347a503a412a0e520ba88ee140cc39344c1f9049422bd5439af101eeabcbe3"}
Oct 30 13:13:14.972911 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:14.972818    4943 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-w4-88f6a4-7dd558b944-gkzck"
Oct 30 13:13:14.982176 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:14.982154    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w3-d0a18c-778df4584c-4f9hx"]
Oct 30 13:13:14.983967 ip-10-0-142-84 kubenswrapper[4943]: W1030 13:13:14.983936    4943 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod93220213_580a_465b_b060_3cea89bfd247.slice/crio-8ddcadab29de337657118a5fe3a8c771a50b6d3f84f950ecd2ed5fe6f52c198a WatchSource:0}: Error finding container 8ddcadab29de337657118a5fe3a8c771a50b6d3f84f950ecd2ed5fe6f52c198a: Status 404 returned error can't find the container with id 8ddcadab29de337657118a5fe3a8c771a50b6d3f84f950ecd2ed5fe6f52c198a
Oct 30 13:13:15.034447 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:15.034405    4943 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-w4-88f6a4-7dd558b944-gkzck" podStartSLOduration=2.034391098 podStartE2EDuration="2.034391098s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:13:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:13:15.032515443 +0000 UTC m=+970.820519174" watchObservedRunningTime="2026-10-30 13:13:15.034391098 +0000 UTC m=+970.822394828"
Oct 30 13:13:15.557091 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:15.557054    4943 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w6-ab3176-5f84755579-6t85j"]
Oct 30 13:13:15.560117 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:15.560093    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-ab3176-5f84755579-6t85j"
Oct 30 13:13:15.578838 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:15.578812    4943 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w6-ab3176\""
Oct 30 13:13:15.578838 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:15.578828    4943 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w6-ab3176-dockercfg-8bx8s\""
Oct 30 13:13:15.581498 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:15.581472    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w6-ab3176-5f84755579-6t85j"]
Oct 30 13:13:15.704286 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:15.703324    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wwstb\" (UniqueName: \"kubernetes.io/projected/1d807dba-b7c2-412d-8be6-aa6a4f6db610-kube-api-access-wwstb\") pod \"payload-pre-processing-e2e-worker-w6-ab3176-5f84755579-6t85j\" (UID: \"1d807dba-b7c2-412d-8be6-aa6a4f6db610\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-ab3176-5f84755579-6t85j"
Oct 30 13:13:15.704286 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:15.703478    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1d807dba-b7c2-412d-8be6-aa6a4f6db610-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w6-ab3176-5f84755579-6t85j\" (UID: \"1d807dba-b7c2-412d-8be6-aa6a4f6db610\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-ab3176-5f84755579-6t85j"
Oct 30 13:13:15.804282 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:15.804221    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1d807dba-b7c2-412d-8be6-aa6a4f6db610-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w6-ab3176-5f84755579-6t85j\" (UID: \"1d807dba-b7c2-412d-8be6-aa6a4f6db610\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-ab3176-5f84755579-6t85j"
Oct 30 13:13:15.804476 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:15.804330    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wwstb\" (UniqueName: \"kubernetes.io/projected/1d807dba-b7c2-412d-8be6-aa6a4f6db610-kube-api-access-wwstb\") pod \"payload-pre-processing-e2e-worker-w6-ab3176-5f84755579-6t85j\" (UID: \"1d807dba-b7c2-412d-8be6-aa6a4f6db610\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-ab3176-5f84755579-6t85j"
Oct 30 13:13:15.805021 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:15.804995    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1d807dba-b7c2-412d-8be6-aa6a4f6db610-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w6-ab3176-5f84755579-6t85j\" (UID: \"1d807dba-b7c2-412d-8be6-aa6a4f6db610\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-ab3176-5f84755579-6t85j"
Oct 30 13:13:15.831773 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:15.831710    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wwstb\" (UniqueName: \"kubernetes.io/projected/1d807dba-b7c2-412d-8be6-aa6a4f6db610-kube-api-access-wwstb\") pod \"payload-pre-processing-e2e-worker-w6-ab3176-5f84755579-6t85j\" (UID: \"1d807dba-b7c2-412d-8be6-aa6a4f6db610\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-ab3176-5f84755579-6t85j"
Oct 30 13:13:15.872412 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:15.872388    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-ab3176-5f84755579-6t85j"
Oct 30 13:13:15.981552 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:15.981520    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-d0a18c-778df4584c-4f9hx" event={"ID":"93220213-580a-465b-b060-3cea89bfd247","Type":"ContainerStarted","Data":"c02e281bfa1ce0adc4e192e59fa3f1afa783120d764a61804c6e5e1680cdd7bf"}
Oct 30 13:13:15.981707 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:15.981580    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-d0a18c-778df4584c-4f9hx" event={"ID":"93220213-580a-465b-b060-3cea89bfd247","Type":"ContainerStarted","Data":"8ddcadab29de337657118a5fe3a8c771a50b6d3f84f950ecd2ed5fe6f52c198a"}
Oct 30 13:13:15.982434 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:15.982411    4943 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-d0a18c-778df4584c-4f9hx"
Oct 30 13:13:16.060420 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:16.060373    4943 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-d0a18c-778df4584c-4f9hx" podStartSLOduration=2.060359183 podStartE2EDuration="2.060359183s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:13:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:13:16.059205666 +0000 UTC m=+971.847209398" watchObservedRunningTime="2026-10-30 13:13:16.060359183 +0000 UTC m=+971.848362910"
Oct 30 13:13:16.228146 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:16.228110    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w6-ab3176-5f84755579-6t85j"]
Oct 30 13:13:16.231581 ip-10-0-142-84 kubenswrapper[4943]: W1030 13:13:16.231543    4943 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1d807dba_b7c2_412d_8be6_aa6a4f6db610.slice/crio-2bf03f26f8cd94649bd28b9422993ebcc32a121da79aa4b78a2395621e7dc175 WatchSource:0}: Error finding container 2bf03f26f8cd94649bd28b9422993ebcc32a121da79aa4b78a2395621e7dc175: Status 404 returned error can't find the container with id 2bf03f26f8cd94649bd28b9422993ebcc32a121da79aa4b78a2395621e7dc175
Oct 30 13:13:16.989362 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:16.989326    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-ab3176-5f84755579-6t85j" event={"ID":"1d807dba-b7c2-412d-8be6-aa6a4f6db610","Type":"ContainerStarted","Data":"f9d2a89a5d0a684bffda22e5c8c2deb921f0b76ad3737c2dcc6ea495660b3321"}
Oct 30 13:13:16.989861 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:16.989385    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-ab3176-5f84755579-6t85j" event={"ID":"1d807dba-b7c2-412d-8be6-aa6a4f6db610","Type":"ContainerStarted","Data":"2bf03f26f8cd94649bd28b9422993ebcc32a121da79aa4b78a2395621e7dc175"}
Oct 30 13:13:16.989861 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:16.989416    4943 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-ab3176-5f84755579-6t85j"
Oct 30 13:13:17.043587 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:17.043537    4943 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-ab3176-5f84755579-6t85j" podStartSLOduration=2.043523695 podStartE2EDuration="2.043523695s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:13:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:13:17.040500988 +0000 UTC m=+972.828504718" watchObservedRunningTime="2026-10-30 13:13:17.043523695 +0000 UTC m=+972.831527425"
Oct 30 13:13:18.458700 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:18.458653    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-hdmrm"]
Oct 30 13:13:18.658365 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:18.658327    4943 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w1-05942d-7d456fbd84-mvdr9"]
Oct 30 13:13:18.661557 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:18.661537    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w1-05942d-7d456fbd84-mvdr9"
Oct 30 13:13:18.715334 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:18.705790    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w1-05942d-7d456fbd84-mvdr9"]
Oct 30 13:13:18.715334 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:18.709975    4943 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w1-05942d\""
Oct 30 13:13:18.715334 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:18.710164    4943 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w1-05942d-dockercfg-2h8s9\""
Oct 30 13:13:18.841605 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:18.841569    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7607a99a-237c-4d0c-b5d0-5168ec6a6e3f-plugins-config-volume\") pod \"payload-processing-e2e-worker-w1-05942d-7d456fbd84-mvdr9\" (UID: \"7607a99a-237c-4d0c-b5d0-5168ec6a6e3f\") " pod="openshift-ingress/payload-processing-e2e-worker-w1-05942d-7d456fbd84-mvdr9"
Oct 30 13:13:18.841775 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:18.841622    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ncqth\" (UniqueName: \"kubernetes.io/projected/7607a99a-237c-4d0c-b5d0-5168ec6a6e3f-kube-api-access-ncqth\") pod \"payload-processing-e2e-worker-w1-05942d-7d456fbd84-mvdr9\" (UID: \"7607a99a-237c-4d0c-b5d0-5168ec6a6e3f\") " pod="openshift-ingress/payload-processing-e2e-worker-w1-05942d-7d456fbd84-mvdr9"
Oct 30 13:13:18.942603 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:18.942563    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7607a99a-237c-4d0c-b5d0-5168ec6a6e3f-plugins-config-volume\") pod \"payload-processing-e2e-worker-w1-05942d-7d456fbd84-mvdr9\" (UID: \"7607a99a-237c-4d0c-b5d0-5168ec6a6e3f\") " pod="openshift-ingress/payload-processing-e2e-worker-w1-05942d-7d456fbd84-mvdr9"
Oct 30 13:13:18.942603 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:18.942605    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ncqth\" (UniqueName: \"kubernetes.io/projected/7607a99a-237c-4d0c-b5d0-5168ec6a6e3f-kube-api-access-ncqth\") pod \"payload-processing-e2e-worker-w1-05942d-7d456fbd84-mvdr9\" (UID: \"7607a99a-237c-4d0c-b5d0-5168ec6a6e3f\") " pod="openshift-ingress/payload-processing-e2e-worker-w1-05942d-7d456fbd84-mvdr9"
Oct 30 13:13:18.943314 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:18.943292    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7607a99a-237c-4d0c-b5d0-5168ec6a6e3f-plugins-config-volume\") pod \"payload-processing-e2e-worker-w1-05942d-7d456fbd84-mvdr9\" (UID: \"7607a99a-237c-4d0c-b5d0-5168ec6a6e3f\") " pod="openshift-ingress/payload-processing-e2e-worker-w1-05942d-7d456fbd84-mvdr9"
Oct 30 13:13:18.968697 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:18.968640    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ncqth\" (UniqueName: \"kubernetes.io/projected/7607a99a-237c-4d0c-b5d0-5168ec6a6e3f-kube-api-access-ncqth\") pod \"payload-processing-e2e-worker-w1-05942d-7d456fbd84-mvdr9\" (UID: \"7607a99a-237c-4d0c-b5d0-5168ec6a6e3f\") " pod="openshift-ingress/payload-processing-e2e-worker-w1-05942d-7d456fbd84-mvdr9"
Oct 30 13:13:18.974572 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:18.974554    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w1-05942d-7d456fbd84-mvdr9"
Oct 30 13:13:19.358310 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:19.358273    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w1-05942d-7d456fbd84-mvdr9"]
Oct 30 13:13:19.359665 ip-10-0-142-84 kubenswrapper[4943]: W1030 13:13:19.359639    4943 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7607a99a_237c_4d0c_b5d0_5168ec6a6e3f.slice/crio-cc046983286ad8903c5751524e4c09f74eda7e4ae7f7f7022b105aa95e74f83a WatchSource:0}: Error finding container cc046983286ad8903c5751524e4c09f74eda7e4ae7f7f7022b105aa95e74f83a: Status 404 returned error can't find the container with id cc046983286ad8903c5751524e4c09f74eda7e4ae7f7f7022b105aa95e74f83a
Oct 30 13:13:20.020233 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:20.020196    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w1-05942d-7d456fbd84-mvdr9" event={"ID":"7607a99a-237c-4d0c-b5d0-5168ec6a6e3f","Type":"ContainerStarted","Data":"811b2cfece4fbf41e6834879aa348dada91ef3cafecf7876a28a4cf2b00d8d30"}
Oct 30 13:13:20.020644 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:20.020281    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w1-05942d-7d456fbd84-mvdr9" event={"ID":"7607a99a-237c-4d0c-b5d0-5168ec6a6e3f","Type":"ContainerStarted","Data":"cc046983286ad8903c5751524e4c09f74eda7e4ae7f7f7022b105aa95e74f83a"}
Oct 30 13:13:20.020644 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:20.020311    4943 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-w1-05942d-7d456fbd84-mvdr9"
Oct 30 13:13:20.085033 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:20.084969    4943 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-w1-05942d-7d456fbd84-mvdr9" podStartSLOduration=2.084952924 podStartE2EDuration="2.084952924s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:13:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:13:20.084848149 +0000 UTC m=+975.872851891" watchObservedRunningTime="2026-10-30 13:13:20.084952924 +0000 UTC m=+975.872956656"
Oct 30 13:13:23.603716 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:23.603665    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-hdmrm"]
Oct 30 13:13:23.996028 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:23.995984    4943 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-w0-031737-57d4df6b68-xr5s2"
Oct 30 13:13:24.993352 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:24.993315    4943 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w2-f76c09-5bdd4bf999-gbp8q"]
Oct 30 13:13:25.015515 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:24.998442    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-f76c09-5bdd4bf999-gbp8q"
Oct 30 13:13:25.036283 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:25.034572    4943 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w2-f76c09-dockercfg-tthxq\""
Oct 30 13:13:25.036283 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:25.035287    4943 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w2-f76c09\""
Oct 30 13:13:25.054977 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:25.053178    4943 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w2-f76c09-588d998d5b-z94mv"]
Oct 30 13:13:25.083855 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:25.083826    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w2-f76c09-5bdd4bf999-gbp8q"]
Oct 30 13:13:25.084201 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:25.084185    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w2-f76c09-588d998d5b-z94mv"]
Oct 30 13:13:25.084364 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:25.084133    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w2-f76c09-588d998d5b-z94mv"
Oct 30 13:13:25.141301 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:25.141241    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f35ce2f9-b769-4ec4-9c57-a31057339d93-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w2-f76c09-5bdd4bf999-gbp8q\" (UID: \"f35ce2f9-b769-4ec4-9c57-a31057339d93\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-f76c09-5bdd4bf999-gbp8q"
Oct 30 13:13:25.141476 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:25.141417    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d6rm7\" (UniqueName: \"kubernetes.io/projected/f35ce2f9-b769-4ec4-9c57-a31057339d93-kube-api-access-d6rm7\") pod \"payload-pre-processing-e2e-worker-w2-f76c09-5bdd4bf999-gbp8q\" (UID: \"f35ce2f9-b769-4ec4-9c57-a31057339d93\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-f76c09-5bdd4bf999-gbp8q"
Oct 30 13:13:25.242542 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:25.242504    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d6rm7\" (UniqueName: \"kubernetes.io/projected/f35ce2f9-b769-4ec4-9c57-a31057339d93-kube-api-access-d6rm7\") pod \"payload-pre-processing-e2e-worker-w2-f76c09-5bdd4bf999-gbp8q\" (UID: \"f35ce2f9-b769-4ec4-9c57-a31057339d93\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-f76c09-5bdd4bf999-gbp8q"
Oct 30 13:13:25.242745 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:25.242658    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5wd78\" (UniqueName: \"kubernetes.io/projected/278a0939-7f8a-4551-bac6-a4ef6b5f44a9-kube-api-access-5wd78\") pod \"payload-processing-e2e-worker-w2-f76c09-588d998d5b-z94mv\" (UID: \"278a0939-7f8a-4551-bac6-a4ef6b5f44a9\") " pod="openshift-ingress/payload-processing-e2e-worker-w2-f76c09-588d998d5b-z94mv"
Oct 30 13:13:25.242745 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:25.242696    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f35ce2f9-b769-4ec4-9c57-a31057339d93-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w2-f76c09-5bdd4bf999-gbp8q\" (UID: \"f35ce2f9-b769-4ec4-9c57-a31057339d93\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-f76c09-5bdd4bf999-gbp8q"
Oct 30 13:13:25.242879 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:25.242800    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/278a0939-7f8a-4551-bac6-a4ef6b5f44a9-plugins-config-volume\") pod \"payload-processing-e2e-worker-w2-f76c09-588d998d5b-z94mv\" (UID: \"278a0939-7f8a-4551-bac6-a4ef6b5f44a9\") " pod="openshift-ingress/payload-processing-e2e-worker-w2-f76c09-588d998d5b-z94mv"
Oct 30 13:13:25.243430 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:25.243383    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f35ce2f9-b769-4ec4-9c57-a31057339d93-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w2-f76c09-5bdd4bf999-gbp8q\" (UID: \"f35ce2f9-b769-4ec4-9c57-a31057339d93\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-f76c09-5bdd4bf999-gbp8q"
Oct 30 13:13:25.287022 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:25.286985    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d6rm7\" (UniqueName: \"kubernetes.io/projected/f35ce2f9-b769-4ec4-9c57-a31057339d93-kube-api-access-d6rm7\") pod \"payload-pre-processing-e2e-worker-w2-f76c09-5bdd4bf999-gbp8q\" (UID: \"f35ce2f9-b769-4ec4-9c57-a31057339d93\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-f76c09-5bdd4bf999-gbp8q"
Oct 30 13:13:25.344004 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:25.343962    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/278a0939-7f8a-4551-bac6-a4ef6b5f44a9-plugins-config-volume\") pod \"payload-processing-e2e-worker-w2-f76c09-588d998d5b-z94mv\" (UID: \"278a0939-7f8a-4551-bac6-a4ef6b5f44a9\") " pod="openshift-ingress/payload-processing-e2e-worker-w2-f76c09-588d998d5b-z94mv"
Oct 30 13:13:25.344149 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:25.344125    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5wd78\" (UniqueName: \"kubernetes.io/projected/278a0939-7f8a-4551-bac6-a4ef6b5f44a9-kube-api-access-5wd78\") pod \"payload-processing-e2e-worker-w2-f76c09-588d998d5b-z94mv\" (UID: \"278a0939-7f8a-4551-bac6-a4ef6b5f44a9\") " pod="openshift-ingress/payload-processing-e2e-worker-w2-f76c09-588d998d5b-z94mv"
Oct 30 13:13:25.344680 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:25.344659    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/278a0939-7f8a-4551-bac6-a4ef6b5f44a9-plugins-config-volume\") pod \"payload-processing-e2e-worker-w2-f76c09-588d998d5b-z94mv\" (UID: \"278a0939-7f8a-4551-bac6-a4ef6b5f44a9\") " pod="openshift-ingress/payload-processing-e2e-worker-w2-f76c09-588d998d5b-z94mv"
Oct 30 13:13:25.386884 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:25.386840    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5wd78\" (UniqueName: \"kubernetes.io/projected/278a0939-7f8a-4551-bac6-a4ef6b5f44a9-kube-api-access-5wd78\") pod \"payload-processing-e2e-worker-w2-f76c09-588d998d5b-z94mv\" (UID: \"278a0939-7f8a-4551-bac6-a4ef6b5f44a9\") " pod="openshift-ingress/payload-processing-e2e-worker-w2-f76c09-588d998d5b-z94mv"
Oct 30 13:13:25.387076 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:25.387015    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-f76c09-5bdd4bf999-gbp8q"
Oct 30 13:13:25.411351 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:25.411318    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w2-f76c09-588d998d5b-z94mv"
Oct 30 13:13:25.695151 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:25.695125    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w2-f76c09-5bdd4bf999-gbp8q"]
Oct 30 13:13:25.696917 ip-10-0-142-84 kubenswrapper[4943]: W1030 13:13:25.696879    4943 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf35ce2f9_b769_4ec4_9c57_a31057339d93.slice/crio-d7e07a217a15b26e7ab1e5db8e20d35dd7cb51a93197af04904d5dfce1bf0221 WatchSource:0}: Error finding container d7e07a217a15b26e7ab1e5db8e20d35dd7cb51a93197af04904d5dfce1bf0221: Status 404 returned error can't find the container with id d7e07a217a15b26e7ab1e5db8e20d35dd7cb51a93197af04904d5dfce1bf0221
Oct 30 13:13:25.758021 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:25.757991    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w2-f76c09-588d998d5b-z94mv"]
Oct 30 13:13:25.759130 ip-10-0-142-84 kubenswrapper[4943]: W1030 13:13:25.759108    4943 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod278a0939_7f8a_4551_bac6_a4ef6b5f44a9.slice/crio-d739aa38c7b9121bc9775ee4aa10cb488e5faae26be150f58c21a1fdb0ea6429 WatchSource:0}: Error finding container d739aa38c7b9121bc9775ee4aa10cb488e5faae26be150f58c21a1fdb0ea6429: Status 404 returned error can't find the container with id d739aa38c7b9121bc9775ee4aa10cb488e5faae26be150f58c21a1fdb0ea6429
Oct 30 13:13:25.958281 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:25.955069    4943 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w5-f8a55e-697b9bfd75-pcr7h"]
Oct 30 13:13:25.973613 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:25.973580    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-f8a55e-697b9bfd75-pcr7h"
Oct 30 13:13:25.997295 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:25.995872    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w5-f8a55e-697b9bfd75-pcr7h"]
Oct 30 13:13:26.003719 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:26.002869    4943 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w5-f8a55e\""
Oct 30 13:13:26.010120 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:26.009986    4943 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-w4-88f6a4-7dd558b944-gkzck"
Oct 30 13:13:26.056010 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:26.055876    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/93484267-ce16-4ae2-b52b-e46fae5e0121-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w5-f8a55e-697b9bfd75-pcr7h\" (UID: \"93484267-ce16-4ae2-b52b-e46fae5e0121\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-f8a55e-697b9bfd75-pcr7h"
Oct 30 13:13:26.056010 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:26.055951    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hfqwq\" (UniqueName: \"kubernetes.io/projected/93484267-ce16-4ae2-b52b-e46fae5e0121-kube-api-access-hfqwq\") pod \"payload-pre-processing-e2e-worker-w5-f8a55e-697b9bfd75-pcr7h\" (UID: \"93484267-ce16-4ae2-b52b-e46fae5e0121\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-f8a55e-697b9bfd75-pcr7h"
Oct 30 13:13:26.099347 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:26.099299    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-f76c09-5bdd4bf999-gbp8q" event={"ID":"f35ce2f9-b769-4ec4-9c57-a31057339d93","Type":"ContainerStarted","Data":"74da8e5e76cf9d27e72eb45e48ad35c66ed936f0c10713c1c015d00666116b29"}
Oct 30 13:13:26.099531 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:26.099370    4943 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-f76c09-5bdd4bf999-gbp8q"
Oct 30 13:13:26.099531 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:26.099387    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-f76c09-5bdd4bf999-gbp8q" event={"ID":"f35ce2f9-b769-4ec4-9c57-a31057339d93","Type":"ContainerStarted","Data":"d7e07a217a15b26e7ab1e5db8e20d35dd7cb51a93197af04904d5dfce1bf0221"}
Oct 30 13:13:26.100868 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:26.100836    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w2-f76c09-588d998d5b-z94mv" event={"ID":"278a0939-7f8a-4551-bac6-a4ef6b5f44a9","Type":"ContainerStarted","Data":"961bb93619aa61ec76505cd093319d4b9f87decb2de092205a0d403e678f5366"}
Oct 30 13:13:26.100990 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:26.100884    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w2-f76c09-588d998d5b-z94mv" event={"ID":"278a0939-7f8a-4551-bac6-a4ef6b5f44a9","Type":"ContainerStarted","Data":"d739aa38c7b9121bc9775ee4aa10cb488e5faae26be150f58c21a1fdb0ea6429"}
Oct 30 13:13:26.100990 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:26.100905    4943 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-w2-f76c09-588d998d5b-z94mv"
Oct 30 13:13:26.157248 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:26.157215    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/93484267-ce16-4ae2-b52b-e46fae5e0121-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w5-f8a55e-697b9bfd75-pcr7h\" (UID: \"93484267-ce16-4ae2-b52b-e46fae5e0121\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-f8a55e-697b9bfd75-pcr7h"
Oct 30 13:13:26.157403 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:26.157286    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hfqwq\" (UniqueName: \"kubernetes.io/projected/93484267-ce16-4ae2-b52b-e46fae5e0121-kube-api-access-hfqwq\") pod \"payload-pre-processing-e2e-worker-w5-f8a55e-697b9bfd75-pcr7h\" (UID: \"93484267-ce16-4ae2-b52b-e46fae5e0121\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-f8a55e-697b9bfd75-pcr7h"
Oct 30 13:13:26.158171 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:26.158151    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/93484267-ce16-4ae2-b52b-e46fae5e0121-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w5-f8a55e-697b9bfd75-pcr7h\" (UID: \"93484267-ce16-4ae2-b52b-e46fae5e0121\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-f8a55e-697b9bfd75-pcr7h"
Oct 30 13:13:26.192421 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:26.192362    4943 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-w2-f76c09-588d998d5b-z94mv" podStartSLOduration=2.19234514 podStartE2EDuration="2.19234514s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:13:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:13:26.188438879 +0000 UTC m=+981.976442609" watchObservedRunningTime="2026-10-30 13:13:26.19234514 +0000 UTC m=+981.980348871"
Oct 30 13:13:26.192959 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:26.192939    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hfqwq\" (UniqueName: \"kubernetes.io/projected/93484267-ce16-4ae2-b52b-e46fae5e0121-kube-api-access-hfqwq\") pod \"payload-pre-processing-e2e-worker-w5-f8a55e-697b9bfd75-pcr7h\" (UID: \"93484267-ce16-4ae2-b52b-e46fae5e0121\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-f8a55e-697b9bfd75-pcr7h"
Oct 30 13:13:26.236622 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:26.236572    4943 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-f76c09-5bdd4bf999-gbp8q" podStartSLOduration=2.23656054 podStartE2EDuration="2.23656054s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:13:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:13:26.234622075 +0000 UTC m=+982.022625819" watchObservedRunningTime="2026-10-30 13:13:26.23656054 +0000 UTC m=+982.024564270"
Oct 30 13:13:26.315845 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:26.315754    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-f8a55e-697b9bfd75-pcr7h"
Oct 30 13:13:26.691306 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:26.690963    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w5-f8a55e-697b9bfd75-pcr7h"]
Oct 30 13:13:27.109571 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:27.109487    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-f8a55e-697b9bfd75-pcr7h" event={"ID":"93484267-ce16-4ae2-b52b-e46fae5e0121","Type":"ContainerStarted","Data":"6391329407a4754f2b839e56094f340331c02341f487db984c4ed9943402c1f3"}
Oct 30 13:13:27.109571 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:27.109544    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-f8a55e-697b9bfd75-pcr7h" event={"ID":"93484267-ce16-4ae2-b52b-e46fae5e0121","Type":"ContainerStarted","Data":"6029f2375e42b5588a6e879f40e2ce3c92ca8aaccd9a202dc80cb173ca6e87e0"}
Oct 30 13:13:27.109571 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:27.109570    4943 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-f8a55e-697b9bfd75-pcr7h"
Oct 30 13:13:27.156312 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:27.156242    4943 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-f8a55e-697b9bfd75-pcr7h" podStartSLOduration=2.156230027 podStartE2EDuration="2.156230027s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:13:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:13:27.154318948 +0000 UTC m=+982.942322678" watchObservedRunningTime="2026-10-30 13:13:27.156230027 +0000 UTC m=+982.944233757"
Oct 30 13:13:27.997616 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:27.997586    4943 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-ab3176-5f84755579-6t85j"
Oct 30 13:13:27.997764 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:27.997665    4943 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-d0a18c-778df4584c-4f9hx"
Oct 30 13:13:29.390505 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:29.390459    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-hdmrm"]
Oct 30 13:13:31.029691 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:31.029659    4943 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-w1-05942d-7d456fbd84-mvdr9"
Oct 30 13:13:35.188485 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:35.188444    4943 generic.go:356] "Generic (PLEG): container finished" podID="7bae0af9-fbeb-4480-9452-1abab86550e8" containerID="c36000632379947fbb591c5b3f97b05877896b6ee819563caf1193313eaab073" exitCode=0
Oct 30 13:13:35.188940 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:35.188511    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-qzz29" event={"ID":"7bae0af9-fbeb-4480-9452-1abab86550e8","Type":"ContainerDied","Data":"c36000632379947fbb591c5b3f97b05877896b6ee819563caf1193313eaab073"}
Oct 30 13:13:35.188940 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:35.188918    4943 scope.go:122] "RemoveContainer" containerID="c36000632379947fbb591c5b3f97b05877896b6ee819563caf1193313eaab073"
Oct 30 13:13:35.190253 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:35.190221    4943 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 13:13:35.190359 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:35.190321    4943 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 13:13:35.190398 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:35.190360    4943 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 13:13:35.823056 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:35.823016    4943 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w5-f8a55e-6955dcb987-7t6p5"]
Oct 30 13:13:35.826841 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:35.826817    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-f8a55e-6955dcb987-7t6p5"
Oct 30 13:13:35.850567 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:35.850544    4943 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Oct 30 13:13:35.899689 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:35.895609    4943 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w5-f8a55e-serving-cert\""
Oct 30 13:13:35.899689 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:35.895682    4943 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-5qrbm\""
Oct 30 13:13:35.899689 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:35.899624    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w5-f8a55e-6955dcb987-7t6p5"]
Oct 30 13:13:35.916658 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:35.916625    4943 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Oct 30 13:13:35.916820 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:35.916689    4943 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Oct 30 13:13:35.931610 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:35.931576    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/36d9a0d7-fa1d-463f-a424-3d36606422f3-metrics-certs\") pod \"maas-api-e2e-worker-w5-f8a55e-6955dcb987-7t6p5\" (UID: \"36d9a0d7-fa1d-463f-a424-3d36606422f3\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-f8a55e-6955dcb987-7t6p5"
Oct 30 13:13:35.931834 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:35.931774    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/36d9a0d7-fa1d-463f-a424-3d36606422f3-maas-api-tls\") pod \"maas-api-e2e-worker-w5-f8a55e-6955dcb987-7t6p5\" (UID: \"36d9a0d7-fa1d-463f-a424-3d36606422f3\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-f8a55e-6955dcb987-7t6p5"
Oct 30 13:13:35.931939 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:35.931900    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-29jnk\" (UniqueName: \"kubernetes.io/projected/36d9a0d7-fa1d-463f-a424-3d36606422f3-kube-api-access-29jnk\") pod \"maas-api-e2e-worker-w5-f8a55e-6955dcb987-7t6p5\" (UID: \"36d9a0d7-fa1d-463f-a424-3d36606422f3\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-f8a55e-6955dcb987-7t6p5"
Oct 30 13:13:35.932614 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:35.932585    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/36d9a0d7-fa1d-463f-a424-3d36606422f3-openshift-service-ca\") pod \"maas-api-e2e-worker-w5-f8a55e-6955dcb987-7t6p5\" (UID: \"36d9a0d7-fa1d-463f-a424-3d36606422f3\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-f8a55e-6955dcb987-7t6p5"
Oct 30 13:13:35.987475 ip-10-0-142-84 kubenswrapper[4943]: E1030 13:13:35.987437    4943 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-knqz6" containerName="extractor"
Oct 30 13:13:36.033671 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:36.033637    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/36d9a0d7-fa1d-463f-a424-3d36606422f3-openshift-service-ca\") pod \"maas-api-e2e-worker-w5-f8a55e-6955dcb987-7t6p5\" (UID: \"36d9a0d7-fa1d-463f-a424-3d36606422f3\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-f8a55e-6955dcb987-7t6p5"
Oct 30 13:13:36.033859 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:36.033680    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/36d9a0d7-fa1d-463f-a424-3d36606422f3-metrics-certs\") pod \"maas-api-e2e-worker-w5-f8a55e-6955dcb987-7t6p5\" (UID: \"36d9a0d7-fa1d-463f-a424-3d36606422f3\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-f8a55e-6955dcb987-7t6p5"
Oct 30 13:13:36.033859 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:36.033773    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/36d9a0d7-fa1d-463f-a424-3d36606422f3-maas-api-tls\") pod \"maas-api-e2e-worker-w5-f8a55e-6955dcb987-7t6p5\" (UID: \"36d9a0d7-fa1d-463f-a424-3d36606422f3\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-f8a55e-6955dcb987-7t6p5"
Oct 30 13:13:36.033859 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:36.033807    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-29jnk\" (UniqueName: \"kubernetes.io/projected/36d9a0d7-fa1d-463f-a424-3d36606422f3-kube-api-access-29jnk\") pod \"maas-api-e2e-worker-w5-f8a55e-6955dcb987-7t6p5\" (UID: \"36d9a0d7-fa1d-463f-a424-3d36606422f3\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-f8a55e-6955dcb987-7t6p5"
Oct 30 13:13:36.034426 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:36.034400    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/36d9a0d7-fa1d-463f-a424-3d36606422f3-openshift-service-ca\") pod \"maas-api-e2e-worker-w5-f8a55e-6955dcb987-7t6p5\" (UID: \"36d9a0d7-fa1d-463f-a424-3d36606422f3\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-f8a55e-6955dcb987-7t6p5"
Oct 30 13:13:36.036187 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:36.036161    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/36d9a0d7-fa1d-463f-a424-3d36606422f3-metrics-certs\") pod \"maas-api-e2e-worker-w5-f8a55e-6955dcb987-7t6p5\" (UID: \"36d9a0d7-fa1d-463f-a424-3d36606422f3\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-f8a55e-6955dcb987-7t6p5"
Oct 30 13:13:36.036340 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:36.036222    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/36d9a0d7-fa1d-463f-a424-3d36606422f3-maas-api-tls\") pod \"maas-api-e2e-worker-w5-f8a55e-6955dcb987-7t6p5\" (UID: \"36d9a0d7-fa1d-463f-a424-3d36606422f3\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-f8a55e-6955dcb987-7t6p5"
Oct 30 13:13:36.109765 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:36.109677    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-29jnk\" (UniqueName: \"kubernetes.io/projected/36d9a0d7-fa1d-463f-a424-3d36606422f3-kube-api-access-29jnk\") pod \"maas-api-e2e-worker-w5-f8a55e-6955dcb987-7t6p5\" (UID: \"36d9a0d7-fa1d-463f-a424-3d36606422f3\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-f8a55e-6955dcb987-7t6p5"
Oct 30 13:13:36.139829 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:36.139793    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-f8a55e-6955dcb987-7t6p5"
Oct 30 13:13:36.212574 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:36.212117    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-qzz29" event={"ID":"7bae0af9-fbeb-4480-9452-1abab86550e8","Type":"ContainerStarted","Data":"66b125e7d65ecd902d1fc405dc5610d87fe7ddca1c374f5c597feab53dd0dba7"}
Oct 30 13:13:36.488419 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:36.488374    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w5-f8a55e-6955dcb987-7t6p5"]
Oct 30 13:13:36.491530 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:36.490984    4943 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-qzz29"
Oct 30 13:13:36.492581 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:36.492555    4943 patch_prober.go:29] interesting pod/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-qzz29 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" start-of-body=
Oct 30 13:13:36.492725 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:36.492616    4943 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-qzz29" podUID="7bae0af9-fbeb-4480-9452-1abab86550e8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused"
Oct 30 13:13:36.493338 ip-10-0-142-84 kubenswrapper[4943]: W1030 13:13:36.493306    4943 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod36d9a0d7_fa1d_463f_a424_3d36606422f3.slice/crio-a5d4c350469a958520dd4d3a8ea07aa9a1bfde26004c7c3c73a4c6459ea4ee03 WatchSource:0}: Error finding container a5d4c350469a958520dd4d3a8ea07aa9a1bfde26004c7c3c73a4c6459ea4ee03: Status 404 returned error can't find the container with id a5d4c350469a958520dd4d3a8ea07aa9a1bfde26004c7c3c73a4c6459ea4ee03
Oct 30 13:13:36.494955 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:36.494937    4943 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 13:13:37.121150 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:37.121117    4943 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-w2-f76c09-588d998d5b-z94mv"
Oct 30 13:13:37.125787 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:37.125758    4943 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-f76c09-5bdd4bf999-gbp8q"
Oct 30 13:13:37.229442 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:37.229331    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-f8a55e-6955dcb987-7t6p5" event={"ID":"36d9a0d7-fa1d-463f-a424-3d36606422f3","Type":"ContainerStarted","Data":"a5d4c350469a958520dd4d3a8ea07aa9a1bfde26004c7c3c73a4c6459ea4ee03"}
Oct 30 13:13:37.479398 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:37.478646    4943 patch_prober.go:29] interesting pod/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-qzz29 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" start-of-body=
Oct 30 13:13:37.479398 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:37.478723    4943 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-qzz29" podUID="7bae0af9-fbeb-4480-9452-1abab86550e8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused"
Oct 30 13:13:37.922512 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:37.922426    4943 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-qzz29"
Oct 30 13:13:38.122498 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:38.122465    4943 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-f8a55e-697b9bfd75-pcr7h"
Oct 30 13:13:38.482286 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:38.476513    4943 patch_prober.go:29] interesting pod/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-qzz29 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" start-of-body=
Oct 30 13:13:38.482286 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:38.476574    4943 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-qzz29" podUID="7bae0af9-fbeb-4480-9452-1abab86550e8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused"
Oct 30 13:13:38.830333 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:38.829547    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-hdmrm"]
Oct 30 13:13:38.992309 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:38.991363    4943 patch_prober.go:29] interesting pod/e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-l4l9h container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 13:13:38.992309 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:38.991446    4943 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-l4l9h" podUID="e9509589-96c1-4883-a552-86fd112c147f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:13:39.503287 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:39.502697    4943 patch_prober.go:29] interesting pod/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-qzz29 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" start-of-body=
Oct 30 13:13:39.503287 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:39.502764    4943 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-qzz29" podUID="7bae0af9-fbeb-4480-9452-1abab86550e8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused"
Oct 30 13:13:40.481285 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:40.478631    4943 patch_prober.go:29] interesting pod/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-qzz29 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" start-of-body=
Oct 30 13:13:40.481285 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:40.478705    4943 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-qzz29" podUID="7bae0af9-fbeb-4480-9452-1abab86550e8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused"
Oct 30 13:13:41.487285 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:41.486651    4943 patch_prober.go:29] interesting pod/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-qzz29 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" start-of-body=
Oct 30 13:13:41.487285 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:41.486719    4943 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-qzz29" podUID="7bae0af9-fbeb-4480-9452-1abab86550e8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused"
Oct 30 13:13:42.325253 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:42.325215    4943 generic.go:356] "Generic (PLEG): container finished" podID="e9509589-96c1-4883-a552-86fd112c147f" containerID="8ba842ec132925a085d6d2ecbee87fb111bd4c7e6c15faada2102de24f212d34" exitCode=0
Oct 30 13:13:42.325489 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:42.325292    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-l4l9h" event={"ID":"e9509589-96c1-4883-a552-86fd112c147f","Type":"ContainerDied","Data":"8ba842ec132925a085d6d2ecbee87fb111bd4c7e6c15faada2102de24f212d34"}
Oct 30 13:13:42.325780 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:42.325760    4943 scope.go:122] "RemoveContainer" containerID="8ba842ec132925a085d6d2ecbee87fb111bd4c7e6c15faada2102de24f212d34"
Oct 30 13:13:42.326982 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:42.326958    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-f8a55e-6955dcb987-7t6p5" event={"ID":"36d9a0d7-fa1d-463f-a424-3d36606422f3","Type":"ContainerStarted","Data":"6950212af90232cd183c627313d12524717292cc72163ac7b5d223f5ffc095eb"}
Oct 30 13:13:42.327062 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:42.327011    4943 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-f8a55e-6955dcb987-7t6p5"
Oct 30 13:13:42.327579 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:42.327543    4943 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 13:13:42.327660 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:42.327618    4943 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 13:13:42.327660 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:42.327650    4943 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 13:13:42.447087 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:42.447032    4943 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-f8a55e-6955dcb987-7t6p5" podStartSLOduration=2.693608708 podStartE2EDuration="7.447017338s" totalImagesPullingTime="4.75340863s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:13:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:13:42.444459451 +0000 UTC m=+998.232463181" watchObservedRunningTime="2026-10-30 13:13:42.447017338 +0000 UTC m=+998.235021068"
Oct 30 13:13:42.501777 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:42.501742    4943 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-qzz29"
Oct 30 13:13:42.503758 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:42.503732    4943 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-qzz29"
Oct 30 13:13:43.334871 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:43.334832    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-l4l9h" event={"ID":"e9509589-96c1-4883-a552-86fd112c147f","Type":"ContainerStarted","Data":"9bd8d9a847930fe370a968a15e0e0a65b75b7489b60442efceddc0977539d8e3"}
Oct 30 13:13:44.007676 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:44.007644    4943 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-l4l9h"
Oct 30 13:13:44.008344 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:44.008320    4943 patch_prober.go:29] interesting pod/e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-l4l9h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body=
Oct 30 13:13:44.008510 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:44.008485    4943 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-l4l9h" podUID="e9509589-96c1-4883-a552-86fd112c147f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused"
Oct 30 13:13:44.126846 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:44.125590    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-hdmrm"]
Oct 30 13:13:45.002004 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:45.001967    4943 patch_prober.go:29] interesting pod/e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-l4l9h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body=
Oct 30 13:13:45.002169 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:45.002031    4943 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-l4l9h" podUID="e9509589-96c1-4883-a552-86fd112c147f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused"
Oct 30 13:13:46.002015 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:46.001975    4943 patch_prober.go:29] interesting pod/e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-l4l9h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body=
Oct 30 13:13:46.002567 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:46.002038    4943 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-l4l9h" podUID="e9509589-96c1-4883-a552-86fd112c147f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused"
Oct 30 13:13:47.002802 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:47.002323    4943 patch_prober.go:29] interesting pod/e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-l4l9h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body=
Oct 30 13:13:47.002802 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:47.002400    4943 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-l4l9h" podUID="e9509589-96c1-4883-a552-86fd112c147f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused"
Oct 30 13:13:48.001778 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:48.001743    4943 patch_prober.go:29] interesting pod/e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-l4l9h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body=
Oct 30 13:13:48.001947 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:48.001800    4943 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-l4l9h" podUID="e9509589-96c1-4883-a552-86fd112c147f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused"
Oct 30 13:13:48.340801 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:48.340723    4943 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-f8a55e-6955dcb987-7t6p5"
Oct 30 13:13:49.006756 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:49.006725    4943 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-l4l9h"
Oct 30 13:13:49.007213 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:49.007191    4943 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-l4l9h"
Oct 30 13:13:49.007996 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:49.007973    4943 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-l4l9h"
Oct 30 13:13:50.433206 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:50.433169    4943 generic.go:356] "Generic (PLEG): container finished" podID="8c4e0447-d0df-4536-bea2-00f13e370c5b" containerID="f8327c30b96218ee8c871b6cded44493a4bbe2ef4d8aeacae77de64292f0f756" exitCode=0
Oct 30 13:13:50.433737 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:50.433242    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-fdh7n" event={"ID":"8c4e0447-d0df-4536-bea2-00f13e370c5b","Type":"ContainerDied","Data":"f8327c30b96218ee8c871b6cded44493a4bbe2ef4d8aeacae77de64292f0f756"}
Oct 30 13:13:50.433737 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:50.433734    4943 scope.go:122] "RemoveContainer" containerID="f8327c30b96218ee8c871b6cded44493a4bbe2ef4d8aeacae77de64292f0f756"
Oct 30 13:13:50.435136 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:50.435107    4943 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 13:13:50.435224 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:50.435177    4943 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 13:13:50.435224 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:50.435211    4943 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 13:13:51.440950 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:51.440909    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-fdh7n" event={"ID":"8c4e0447-d0df-4536-bea2-00f13e370c5b","Type":"ContainerStarted","Data":"2bd0afd903ffbbd3b0692efa7b6a8494405b113180956130861bad56145038f8"}
Oct 30 13:13:51.822348 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:51.822241    4943 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-fdh7n"
Oct 30 13:13:51.823597 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:51.823566    4943 patch_prober.go:29] interesting pod/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-fdh7n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" start-of-body=
Oct 30 13:13:51.823723 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:51.823629    4943 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-fdh7n" podUID="8c4e0447-d0df-4536-bea2-00f13e370c5b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused"
Oct 30 13:13:52.821380 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:52.821346    4943 patch_prober.go:29] interesting pod/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-fdh7n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" start-of-body=
Oct 30 13:13:52.821756 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:52.821397    4943 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-fdh7n" podUID="8c4e0447-d0df-4536-bea2-00f13e370c5b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused"
Oct 30 13:13:52.922252 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:52.922222    4943 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-fdh7n"
Oct 30 13:13:53.822233 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:53.822200    4943 patch_prober.go:29] interesting pod/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-fdh7n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" start-of-body=
Oct 30 13:13:53.822624 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:53.822294    4943 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-fdh7n" podUID="8c4e0447-d0df-4536-bea2-00f13e370c5b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused"
Oct 30 13:13:54.359763 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:54.359733    4943 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w6-ab3176/simulator-w6-ab3176-kserve-6d9b64549f-dc8xz"]
Oct 30 13:13:54.368345 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:54.368320    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-ab3176/simulator-w6-ab3176-kserve-6d9b64549f-dc8xz"
Oct 30 13:13:54.394898 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:54.394872    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w6-ab3176/simulator-w6-ab3176-kserve-6d9b64549f-dc8xz"]
Oct 30 13:13:54.405834 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:54.405794    4943 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w6-ab3176\"/\"default-dockercfg-lgbnx\""
Oct 30 13:13:54.406065 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:54.406047    4943 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w6-ab3176\"/\"simulator-w6-ab3176-kserve-self-signed-certs\""
Oct 30 13:13:54.412578 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:54.412549    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-48mt6\" (UniqueName: \"kubernetes.io/projected/da306792-4f80-4df4-aa44-82991ab856e4-kube-api-access-48mt6\") pod \"simulator-w6-ab3176-kserve-6d9b64549f-dc8xz\" (UID: \"da306792-4f80-4df4-aa44-82991ab856e4\") " pod="e2e-models-e2e-worker-w6-ab3176/simulator-w6-ab3176-kserve-6d9b64549f-dc8xz"
Oct 30 13:13:54.412693 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:54.412612    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/da306792-4f80-4df4-aa44-82991ab856e4-home\") pod \"simulator-w6-ab3176-kserve-6d9b64549f-dc8xz\" (UID: \"da306792-4f80-4df4-aa44-82991ab856e4\") " pod="e2e-models-e2e-worker-w6-ab3176/simulator-w6-ab3176-kserve-6d9b64549f-dc8xz"
Oct 30 13:13:54.412771 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:54.412746    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/da306792-4f80-4df4-aa44-82991ab856e4-tls-certs\") pod \"simulator-w6-ab3176-kserve-6d9b64549f-dc8xz\" (UID: \"da306792-4f80-4df4-aa44-82991ab856e4\") " pod="e2e-models-e2e-worker-w6-ab3176/simulator-w6-ab3176-kserve-6d9b64549f-dc8xz"
Oct 30 13:13:54.412829 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:54.412813    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/da306792-4f80-4df4-aa44-82991ab856e4-model-cache\") pod \"simulator-w6-ab3176-kserve-6d9b64549f-dc8xz\" (UID: \"da306792-4f80-4df4-aa44-82991ab856e4\") " pod="e2e-models-e2e-worker-w6-ab3176/simulator-w6-ab3176-kserve-6d9b64549f-dc8xz"
Oct 30 13:13:54.412897 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:54.412881    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/da306792-4f80-4df4-aa44-82991ab856e4-tmp-dir\") pod \"simulator-w6-ab3176-kserve-6d9b64549f-dc8xz\" (UID: \"da306792-4f80-4df4-aa44-82991ab856e4\") " pod="e2e-models-e2e-worker-w6-ab3176/simulator-w6-ab3176-kserve-6d9b64549f-dc8xz"
Oct 30 13:13:54.412955 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:54.412939    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/da306792-4f80-4df4-aa44-82991ab856e4-dshm\") pod \"simulator-w6-ab3176-kserve-6d9b64549f-dc8xz\" (UID: \"da306792-4f80-4df4-aa44-82991ab856e4\") " pod="e2e-models-e2e-worker-w6-ab3176/simulator-w6-ab3176-kserve-6d9b64549f-dc8xz"
Oct 30 13:13:54.420048 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:54.420026    4943 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w6-ab3176\"/\"kube-root-ca.crt\""
Oct 30 13:13:54.420048 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:54.420042    4943 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w6-ab3176\"/\"openshift-service-ca.crt\""
Oct 30 13:13:54.513964 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:54.513927    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-48mt6\" (UniqueName: \"kubernetes.io/projected/da306792-4f80-4df4-aa44-82991ab856e4-kube-api-access-48mt6\") pod \"simulator-w6-ab3176-kserve-6d9b64549f-dc8xz\" (UID: \"da306792-4f80-4df4-aa44-82991ab856e4\") " pod="e2e-models-e2e-worker-w6-ab3176/simulator-w6-ab3176-kserve-6d9b64549f-dc8xz"
Oct 30 13:13:54.514155 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:54.513974    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/da306792-4f80-4df4-aa44-82991ab856e4-home\") pod \"simulator-w6-ab3176-kserve-6d9b64549f-dc8xz\" (UID: \"da306792-4f80-4df4-aa44-82991ab856e4\") " pod="e2e-models-e2e-worker-w6-ab3176/simulator-w6-ab3176-kserve-6d9b64549f-dc8xz"
Oct 30 13:13:54.514155 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:54.514004    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/da306792-4f80-4df4-aa44-82991ab856e4-tls-certs\") pod \"simulator-w6-ab3176-kserve-6d9b64549f-dc8xz\" (UID: \"da306792-4f80-4df4-aa44-82991ab856e4\") " pod="e2e-models-e2e-worker-w6-ab3176/simulator-w6-ab3176-kserve-6d9b64549f-dc8xz"
Oct 30 13:13:54.514155 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:54.514034    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/da306792-4f80-4df4-aa44-82991ab856e4-model-cache\") pod \"simulator-w6-ab3176-kserve-6d9b64549f-dc8xz\" (UID: \"da306792-4f80-4df4-aa44-82991ab856e4\") " pod="e2e-models-e2e-worker-w6-ab3176/simulator-w6-ab3176-kserve-6d9b64549f-dc8xz"
Oct 30 13:13:54.514155 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:54.514068    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/da306792-4f80-4df4-aa44-82991ab856e4-tmp-dir\") pod \"simulator-w6-ab3176-kserve-6d9b64549f-dc8xz\" (UID: \"da306792-4f80-4df4-aa44-82991ab856e4\") " pod="e2e-models-e2e-worker-w6-ab3176/simulator-w6-ab3176-kserve-6d9b64549f-dc8xz"
Oct 30 13:13:54.514155 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:54.514109    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/da306792-4f80-4df4-aa44-82991ab856e4-dshm\") pod \"simulator-w6-ab3176-kserve-6d9b64549f-dc8xz\" (UID: \"da306792-4f80-4df4-aa44-82991ab856e4\") " pod="e2e-models-e2e-worker-w6-ab3176/simulator-w6-ab3176-kserve-6d9b64549f-dc8xz"
Oct 30 13:13:54.514579 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:54.514554    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/da306792-4f80-4df4-aa44-82991ab856e4-tmp-dir\") pod \"simulator-w6-ab3176-kserve-6d9b64549f-dc8xz\" (UID: \"da306792-4f80-4df4-aa44-82991ab856e4\") " pod="e2e-models-e2e-worker-w6-ab3176/simulator-w6-ab3176-kserve-6d9b64549f-dc8xz"
Oct 30 13:13:54.514651 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:54.514556    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/da306792-4f80-4df4-aa44-82991ab856e4-home\") pod \"simulator-w6-ab3176-kserve-6d9b64549f-dc8xz\" (UID: \"da306792-4f80-4df4-aa44-82991ab856e4\") " pod="e2e-models-e2e-worker-w6-ab3176/simulator-w6-ab3176-kserve-6d9b64549f-dc8xz"
Oct 30 13:13:54.514688 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:54.514660    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/da306792-4f80-4df4-aa44-82991ab856e4-model-cache\") pod \"simulator-w6-ab3176-kserve-6d9b64549f-dc8xz\" (UID: \"da306792-4f80-4df4-aa44-82991ab856e4\") " pod="e2e-models-e2e-worker-w6-ab3176/simulator-w6-ab3176-kserve-6d9b64549f-dc8xz"
Oct 30 13:13:54.516441 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:54.516419    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/da306792-4f80-4df4-aa44-82991ab856e4-dshm\") pod \"simulator-w6-ab3176-kserve-6d9b64549f-dc8xz\" (UID: \"da306792-4f80-4df4-aa44-82991ab856e4\") " pod="e2e-models-e2e-worker-w6-ab3176/simulator-w6-ab3176-kserve-6d9b64549f-dc8xz"
Oct 30 13:13:54.516723 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:54.516706    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/da306792-4f80-4df4-aa44-82991ab856e4-tls-certs\") pod \"simulator-w6-ab3176-kserve-6d9b64549f-dc8xz\" (UID: \"da306792-4f80-4df4-aa44-82991ab856e4\") " pod="e2e-models-e2e-worker-w6-ab3176/simulator-w6-ab3176-kserve-6d9b64549f-dc8xz"
Oct 30 13:13:54.544634 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:54.544611    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-48mt6\" (UniqueName: \"kubernetes.io/projected/da306792-4f80-4df4-aa44-82991ab856e4-kube-api-access-48mt6\") pod \"simulator-w6-ab3176-kserve-6d9b64549f-dc8xz\" (UID: \"da306792-4f80-4df4-aa44-82991ab856e4\") " pod="e2e-models-e2e-worker-w6-ab3176/simulator-w6-ab3176-kserve-6d9b64549f-dc8xz"
Oct 30 13:13:54.700990 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:54.700958    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-ab3176/simulator-w6-ab3176-kserve-6d9b64549f-dc8xz"
Oct 30 13:13:54.821588 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:54.821551    4943 patch_prober.go:29] interesting pod/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-fdh7n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" start-of-body=
Oct 30 13:13:54.821752 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:54.821600    4943 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-fdh7n" podUID="8c4e0447-d0df-4536-bea2-00f13e370c5b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused"
Oct 30 13:13:55.040137 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:55.040109    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w6-ab3176/simulator-w6-ab3176-kserve-6d9b64549f-dc8xz"]
Oct 30 13:13:55.041204 ip-10-0-142-84 kubenswrapper[4943]: W1030 13:13:55.041183    4943 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podda306792_4f80_4df4_aa44_82991ab856e4.slice/crio-0ba6be2b1bb09e4a45ed6b724e833540fce2c9c2ffd2501fd75c47bd6afcb4e8 WatchSource:0}: Error finding container 0ba6be2b1bb09e4a45ed6b724e833540fce2c9c2ffd2501fd75c47bd6afcb4e8: Status 404 returned error can't find the container with id 0ba6be2b1bb09e4a45ed6b724e833540fce2c9c2ffd2501fd75c47bd6afcb4e8
Oct 30 13:13:55.488827 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:55.488785    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-ab3176/simulator-w6-ab3176-kserve-6d9b64549f-dc8xz" event={"ID":"da306792-4f80-4df4-aa44-82991ab856e4","Type":"ContainerStarted","Data":"fd93c8c8b8d5bcd559d09f59b456cbaf348baf2bb7739c60ef28ae03c812bc62"}
Oct 30 13:13:55.489025 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:55.488836    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-ab3176/simulator-w6-ab3176-kserve-6d9b64549f-dc8xz" event={"ID":"da306792-4f80-4df4-aa44-82991ab856e4","Type":"ContainerStarted","Data":"0ba6be2b1bb09e4a45ed6b724e833540fce2c9c2ffd2501fd75c47bd6afcb4e8"}
Oct 30 13:13:55.540431 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:55.540371    4943 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w6-ab3176/simulator-w6-ab3176-kserve-6d9b64549f-dc8xz" podStartSLOduration=1.540358056 podStartE2EDuration="1.540358056s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:13:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:13:55.539027528 +0000 UTC m=+1011.327031258" watchObservedRunningTime="2026-10-30 13:13:55.540358056 +0000 UTC m=+1011.328361785"
Oct 30 13:13:55.701647 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:55.701605    4943 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-ab3176/simulator-w6-ab3176-kserve-6d9b64549f-dc8xz"
Oct 30 13:13:55.822054 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:55.821975    4943 patch_prober.go:29] interesting pod/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-fdh7n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" start-of-body=
Oct 30 13:13:55.822054 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:55.822034    4943 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-fdh7n" podUID="8c4e0447-d0df-4536-bea2-00f13e370c5b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused"
Oct 30 13:13:56.875614 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:56.875578    4943 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-fdh7n"
Oct 30 13:13:56.878354 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:56.878321    4943 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-fdh7n"
Oct 30 13:13:57.724223 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:13:57.724187    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-hdmrm"]
Oct 30 13:14:04.701635 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:14:04.701607    4943 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w6-ab3176/simulator-w6-ab3176-kserve-6d9b64549f-dc8xz"
Oct 30 13:14:04.716370 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:14:04.716344    4943 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w6-ab3176/simulator-w6-ab3176-kserve-6d9b64549f-dc8xz"
Oct 30 13:14:05.561656 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:14:05.561619    4943 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-ab3176/simulator-w6-ab3176-kserve-6d9b64549f-dc8xz"
Oct 30 13:14:05.955238 ip-10-0-142-84 kubenswrapper[4943]: E1030 13:14:05.955200    4943 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-knqz6" containerName="extractor"
Oct 30 13:14:09.029624 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:14:09.029587    4943 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-031737/premium-w0-031737-kserve-58d5857cdc-vfnz9"]
Oct 30 13:14:09.033570 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:14:09.033547    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-031737/premium-w0-031737-kserve-58d5857cdc-vfnz9"
Oct 30 13:14:09.064108 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:14:09.064078    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/79b1a638-b019-4c89-b850-01699ecac9ba-model-cache\") pod \"premium-w0-031737-kserve-58d5857cdc-vfnz9\" (UID: \"79b1a638-b019-4c89-b850-01699ecac9ba\") " pod="e2e-models-e2e-worker-w0-031737/premium-w0-031737-kserve-58d5857cdc-vfnz9"
Oct 30 13:14:09.064242 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:14:09.064113    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/79b1a638-b019-4c89-b850-01699ecac9ba-tls-certs\") pod \"premium-w0-031737-kserve-58d5857cdc-vfnz9\" (UID: \"79b1a638-b019-4c89-b850-01699ecac9ba\") " pod="e2e-models-e2e-worker-w0-031737/premium-w0-031737-kserve-58d5857cdc-vfnz9"
Oct 30 13:14:09.064242 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:14:09.064131    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q62fd\" (UniqueName: \"kubernetes.io/projected/79b1a638-b019-4c89-b850-01699ecac9ba-kube-api-access-q62fd\") pod \"premium-w0-031737-kserve-58d5857cdc-vfnz9\" (UID: \"79b1a638-b019-4c89-b850-01699ecac9ba\") " pod="e2e-models-e2e-worker-w0-031737/premium-w0-031737-kserve-58d5857cdc-vfnz9"
Oct 30 13:14:09.064242 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:14:09.064157    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/79b1a638-b019-4c89-b850-01699ecac9ba-tmp-dir\") pod \"premium-w0-031737-kserve-58d5857cdc-vfnz9\" (UID: \"79b1a638-b019-4c89-b850-01699ecac9ba\") " pod="e2e-models-e2e-worker-w0-031737/premium-w0-031737-kserve-58d5857cdc-vfnz9"
Oct 30 13:14:09.064399 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:14:09.064343    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/79b1a638-b019-4c89-b850-01699ecac9ba-dshm\") pod \"premium-w0-031737-kserve-58d5857cdc-vfnz9\" (UID: \"79b1a638-b019-4c89-b850-01699ecac9ba\") " pod="e2e-models-e2e-worker-w0-031737/premium-w0-031737-kserve-58d5857cdc-vfnz9"
Oct 30 13:14:09.064440 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:14:09.064410    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/79b1a638-b019-4c89-b850-01699ecac9ba-home\") pod \"premium-w0-031737-kserve-58d5857cdc-vfnz9\" (UID: \"79b1a638-b019-4c89-b850-01699ecac9ba\") " pod="e2e-models-e2e-worker-w0-031737/premium-w0-031737-kserve-58d5857cdc-vfnz9"
Oct 30 13:14:09.064892 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:14:09.064872    4943 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-031737\"/\"default-dockercfg-dxfjj\""
Oct 30 13:14:09.065224 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:14:09.064894    4943 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-031737\"/\"premium-w0-031737-kserve-self-signed-certs\""
Oct 30 13:14:09.065224 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:14:09.064939    4943 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-031737\"/\"kube-root-ca.crt\""
Oct 30 13:14:09.065224 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:14:09.065116    4943 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-031737\"/\"openshift-service-ca.crt\""
Oct 30 13:14:09.074571 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:14:09.074548    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-031737/premium-w0-031737-kserve-58d5857cdc-vfnz9"]
Oct 30 13:14:09.165350 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:14:09.165318    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/79b1a638-b019-4c89-b850-01699ecac9ba-model-cache\") pod \"premium-w0-031737-kserve-58d5857cdc-vfnz9\" (UID: \"79b1a638-b019-4c89-b850-01699ecac9ba\") " pod="e2e-models-e2e-worker-w0-031737/premium-w0-031737-kserve-58d5857cdc-vfnz9"
Oct 30 13:14:09.165493 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:14:09.165368    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/79b1a638-b019-4c89-b850-01699ecac9ba-tls-certs\") pod \"premium-w0-031737-kserve-58d5857cdc-vfnz9\" (UID: \"79b1a638-b019-4c89-b850-01699ecac9ba\") " pod="e2e-models-e2e-worker-w0-031737/premium-w0-031737-kserve-58d5857cdc-vfnz9"
Oct 30 13:14:09.165493 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:14:09.165388    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q62fd\" (UniqueName: \"kubernetes.io/projected/79b1a638-b019-4c89-b850-01699ecac9ba-kube-api-access-q62fd\") pod \"premium-w0-031737-kserve-58d5857cdc-vfnz9\" (UID: \"79b1a638-b019-4c89-b850-01699ecac9ba\") " pod="e2e-models-e2e-worker-w0-031737/premium-w0-031737-kserve-58d5857cdc-vfnz9"
Oct 30 13:14:09.165493 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:14:09.165414    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/79b1a638-b019-4c89-b850-01699ecac9ba-tmp-dir\") pod \"premium-w0-031737-kserve-58d5857cdc-vfnz9\" (UID: \"79b1a638-b019-4c89-b850-01699ecac9ba\") " pod="e2e-models-e2e-worker-w0-031737/premium-w0-031737-kserve-58d5857cdc-vfnz9"
Oct 30 13:14:09.165612 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:14:09.165556    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/79b1a638-b019-4c89-b850-01699ecac9ba-dshm\") pod \"premium-w0-031737-kserve-58d5857cdc-vfnz9\" (UID: \"79b1a638-b019-4c89-b850-01699ecac9ba\") " pod="e2e-models-e2e-worker-w0-031737/premium-w0-031737-kserve-58d5857cdc-vfnz9"
Oct 30 13:14:09.165669 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:14:09.165656    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/79b1a638-b019-4c89-b850-01699ecac9ba-home\") pod \"premium-w0-031737-kserve-58d5857cdc-vfnz9\" (UID: \"79b1a638-b019-4c89-b850-01699ecac9ba\") " pod="e2e-models-e2e-worker-w0-031737/premium-w0-031737-kserve-58d5857cdc-vfnz9"
Oct 30 13:14:09.165731 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:14:09.165685    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/79b1a638-b019-4c89-b850-01699ecac9ba-model-cache\") pod \"premium-w0-031737-kserve-58d5857cdc-vfnz9\" (UID: \"79b1a638-b019-4c89-b850-01699ecac9ba\") " pod="e2e-models-e2e-worker-w0-031737/premium-w0-031737-kserve-58d5857cdc-vfnz9"
Oct 30 13:14:09.165789 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:14:09.165726    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/79b1a638-b019-4c89-b850-01699ecac9ba-tmp-dir\") pod \"premium-w0-031737-kserve-58d5857cdc-vfnz9\" (UID: \"79b1a638-b019-4c89-b850-01699ecac9ba\") " pod="e2e-models-e2e-worker-w0-031737/premium-w0-031737-kserve-58d5857cdc-vfnz9"
Oct 30 13:14:09.165964 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:14:09.165937    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/79b1a638-b019-4c89-b850-01699ecac9ba-home\") pod \"premium-w0-031737-kserve-58d5857cdc-vfnz9\" (UID: \"79b1a638-b019-4c89-b850-01699ecac9ba\") " pod="e2e-models-e2e-worker-w0-031737/premium-w0-031737-kserve-58d5857cdc-vfnz9"
Oct 30 13:14:09.167632 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:14:09.167609    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/79b1a638-b019-4c89-b850-01699ecac9ba-dshm\") pod \"premium-w0-031737-kserve-58d5857cdc-vfnz9\" (UID: \"79b1a638-b019-4c89-b850-01699ecac9ba\") " pod="e2e-models-e2e-worker-w0-031737/premium-w0-031737-kserve-58d5857cdc-vfnz9"
Oct 30 13:14:09.168028 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:14:09.168006    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/79b1a638-b019-4c89-b850-01699ecac9ba-tls-certs\") pod \"premium-w0-031737-kserve-58d5857cdc-vfnz9\" (UID: \"79b1a638-b019-4c89-b850-01699ecac9ba\") " pod="e2e-models-e2e-worker-w0-031737/premium-w0-031737-kserve-58d5857cdc-vfnz9"
Oct 30 13:14:09.191665 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:14:09.191644    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q62fd\" (UniqueName: \"kubernetes.io/projected/79b1a638-b019-4c89-b850-01699ecac9ba-kube-api-access-q62fd\") pod \"premium-w0-031737-kserve-58d5857cdc-vfnz9\" (UID: \"79b1a638-b019-4c89-b850-01699ecac9ba\") " pod="e2e-models-e2e-worker-w0-031737/premium-w0-031737-kserve-58d5857cdc-vfnz9"
Oct 30 13:14:09.345582 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:14:09.345499    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-031737/premium-w0-031737-kserve-58d5857cdc-vfnz9"
Oct 30 13:14:09.647374 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:14:09.647341    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-031737/premium-w0-031737-kserve-58d5857cdc-vfnz9"]
Oct 30 13:14:09.649040 ip-10-0-142-84 kubenswrapper[4943]: W1030 13:14:09.649005    4943 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod79b1a638_b019_4c89_b850_01699ecac9ba.slice/crio-f1bd293d49314f1ab9a0ae336edf53625ec8316216f85a1e76b8d3ebff85780e WatchSource:0}: Error finding container f1bd293d49314f1ab9a0ae336edf53625ec8316216f85a1e76b8d3ebff85780e: Status 404 returned error can't find the container with id f1bd293d49314f1ab9a0ae336edf53625ec8316216f85a1e76b8d3ebff85780e
Oct 30 13:14:10.210494 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:14:10.210453    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-hdmrm"]
Oct 30 13:14:10.585490 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:14:10.585392    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-031737/premium-w0-031737-kserve-58d5857cdc-vfnz9" event={"ID":"79b1a638-b019-4c89-b850-01699ecac9ba","Type":"ContainerStarted","Data":"8866cbc1f1139e188db329cb54c34b31771bd730a2a1ba37f03bd113a853fbcf"}
Oct 30 13:14:10.585490 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:14:10.585448    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-031737/premium-w0-031737-kserve-58d5857cdc-vfnz9" event={"ID":"79b1a638-b019-4c89-b850-01699ecac9ba","Type":"ContainerStarted","Data":"f1bd293d49314f1ab9a0ae336edf53625ec8316216f85a1e76b8d3ebff85780e"}
Oct 30 13:14:10.634272 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:14:10.634203    4943 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-031737/premium-w0-031737-kserve-58d5857cdc-vfnz9" podStartSLOduration=2.634190076 podStartE2EDuration="2.634190076s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:14:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:14:10.633355834 +0000 UTC m=+1026.421359571" watchObservedRunningTime="2026-10-30 13:14:10.634190076 +0000 UTC m=+1026.422193823"
Oct 30 13:14:11.345719 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:14:11.345682    4943 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-031737/premium-w0-031737-kserve-58d5857cdc-vfnz9"
Oct 30 13:14:19.345845 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:14:19.345760    4943 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-031737/premium-w0-031737-kserve-58d5857cdc-vfnz9"
Oct 30 13:14:19.358650 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:14:19.358623    4943 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-031737/premium-w0-031737-kserve-58d5857cdc-vfnz9"
Oct 30 13:14:19.675103 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:14:19.675029    4943 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-031737/premium-w0-031737-kserve-58d5857cdc-vfnz9"
Oct 30 13:14:20.225842 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:14:20.225355    4943 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-insights/periodic-gathering-lfhqw-rcd8z"]
Oct 30 13:14:20.232024 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:14:20.230455    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-lfhqw-rcd8z"
Oct 30 13:14:20.248609 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:14:20.248582    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/periodic-gathering-lfhqw-rcd8z"]
Oct 30 13:14:20.259765 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:14:20.259734    4943 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-rlrpr\""
Oct 30 13:14:20.279090 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:14:20.279051    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1494fd10-5e14-4d43-aeaf-08e3d779b232-service-ca-bundle\") pod \"periodic-gathering-lfhqw-rcd8z\" (UID: \"1494fd10-5e14-4d43-aeaf-08e3d779b232\") " pod="openshift-insights/periodic-gathering-lfhqw-rcd8z"
Oct 30 13:14:20.279217 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:14:20.279110    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jkdm6\" (UniqueName: \"kubernetes.io/projected/1494fd10-5e14-4d43-aeaf-08e3d779b232-kube-api-access-jkdm6\") pod \"periodic-gathering-lfhqw-rcd8z\" (UID: \"1494fd10-5e14-4d43-aeaf-08e3d779b232\") " pod="openshift-insights/periodic-gathering-lfhqw-rcd8z"
Oct 30 13:14:20.279217 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:14:20.279187    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1494fd10-5e14-4d43-aeaf-08e3d779b232-trusted-ca-bundle\") pod \"periodic-gathering-lfhqw-rcd8z\" (UID: \"1494fd10-5e14-4d43-aeaf-08e3d779b232\") " pod="openshift-insights/periodic-gathering-lfhqw-rcd8z"
Oct 30 13:14:20.279360 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:14:20.279223    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/1494fd10-5e14-4d43-aeaf-08e3d779b232-archives-path\") pod \"periodic-gathering-lfhqw-rcd8z\" (UID: \"1494fd10-5e14-4d43-aeaf-08e3d779b232\") " pod="openshift-insights/periodic-gathering-lfhqw-rcd8z"
Oct 30 13:14:20.380651 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:14:20.380616    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1494fd10-5e14-4d43-aeaf-08e3d779b232-service-ca-bundle\") pod \"periodic-gathering-lfhqw-rcd8z\" (UID: \"1494fd10-5e14-4d43-aeaf-08e3d779b232\") " pod="openshift-insights/periodic-gathering-lfhqw-rcd8z"
Oct 30 13:14:20.381102 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:14:20.380666    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jkdm6\" (UniqueName: \"kubernetes.io/projected/1494fd10-5e14-4d43-aeaf-08e3d779b232-kube-api-access-jkdm6\") pod \"periodic-gathering-lfhqw-rcd8z\" (UID: \"1494fd10-5e14-4d43-aeaf-08e3d779b232\") " pod="openshift-insights/periodic-gathering-lfhqw-rcd8z"
Oct 30 13:14:20.381102 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:14:20.380719    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1494fd10-5e14-4d43-aeaf-08e3d779b232-trusted-ca-bundle\") pod \"periodic-gathering-lfhqw-rcd8z\" (UID: \"1494fd10-5e14-4d43-aeaf-08e3d779b232\") " pod="openshift-insights/periodic-gathering-lfhqw-rcd8z"
Oct 30 13:14:20.381102 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:14:20.380757    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/1494fd10-5e14-4d43-aeaf-08e3d779b232-archives-path\") pod \"periodic-gathering-lfhqw-rcd8z\" (UID: \"1494fd10-5e14-4d43-aeaf-08e3d779b232\") " pod="openshift-insights/periodic-gathering-lfhqw-rcd8z"
Oct 30 13:14:20.381240 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:14:20.381213    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/1494fd10-5e14-4d43-aeaf-08e3d779b232-archives-path\") pod \"periodic-gathering-lfhqw-rcd8z\" (UID: \"1494fd10-5e14-4d43-aeaf-08e3d779b232\") " pod="openshift-insights/periodic-gathering-lfhqw-rcd8z"
Oct 30 13:14:20.381487 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:14:20.381469    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1494fd10-5e14-4d43-aeaf-08e3d779b232-service-ca-bundle\") pod \"periodic-gathering-lfhqw-rcd8z\" (UID: \"1494fd10-5e14-4d43-aeaf-08e3d779b232\") " pod="openshift-insights/periodic-gathering-lfhqw-rcd8z"
Oct 30 13:14:20.381561 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:14:20.381540    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1494fd10-5e14-4d43-aeaf-08e3d779b232-trusted-ca-bundle\") pod \"periodic-gathering-lfhqw-rcd8z\" (UID: \"1494fd10-5e14-4d43-aeaf-08e3d779b232\") " pod="openshift-insights/periodic-gathering-lfhqw-rcd8z"
Oct 30 13:14:20.406833 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:14:20.406811    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jkdm6\" (UniqueName: \"kubernetes.io/projected/1494fd10-5e14-4d43-aeaf-08e3d779b232-kube-api-access-jkdm6\") pod \"periodic-gathering-lfhqw-rcd8z\" (UID: \"1494fd10-5e14-4d43-aeaf-08e3d779b232\") " pod="openshift-insights/periodic-gathering-lfhqw-rcd8z"
Oct 30 13:14:20.544465 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:14:20.544397    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-lfhqw-rcd8z"
Oct 30 13:14:20.850445 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:14:20.850418    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/periodic-gathering-lfhqw-rcd8z"]
Oct 30 13:14:20.851081 ip-10-0-142-84 kubenswrapper[4943]: W1030 13:14:20.851059    4943 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1494fd10_5e14_4d43_aeaf_08e3d779b232.slice/crio-a1166ce031876db5c135a72996c45da9e61c360753e7092a66c63e2ee01c62a9 WatchSource:0}: Error finding container a1166ce031876db5c135a72996c45da9e61c360753e7092a66c63e2ee01c62a9: Status 404 returned error can't find the container with id a1166ce031876db5c135a72996c45da9e61c360753e7092a66c63e2ee01c62a9
Oct 30 13:14:21.281630 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:14:21.281583    4943 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-2zhxj"]
Oct 30 13:14:21.285680 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:14:21.285649    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-2zhxj"
Oct 30 13:14:21.306420 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:14:21.306398    4943 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w3-d0a18c-openshift-default-dockercfg-jqvd8\""
Oct 30 13:14:21.323392 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:14:21.323364    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-2zhxj"]
Oct 30 13:14:21.397347 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:14:21.392068    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cf47ace6-4db7-4896-98a4-496dab0e722f-istio-token\") pod \"e2e-worker-w3-d0a18c-openshift-default-547d57db97-2zhxj\" (UID: \"cf47ace6-4db7-4896-98a4-496dab0e722f\") " pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-2zhxj"
Oct 30 13:14:21.397347 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:14:21.396619    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zp2tx\" (UniqueName: \"kubernetes.io/projected/cf47ace6-4db7-4896-98a4-496dab0e722f-kube-api-access-zp2tx\") pod \"e2e-worker-w3-d0a18c-openshift-default-547d57db97-2zhxj\" (UID: \"cf47ace6-4db7-4896-98a4-496dab0e722f\") " pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-2zhxj"
Oct 30 13:14:21.397347 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:14:21.396674    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cf47ace6-4db7-4896-98a4-496dab0e722f-istiod-ca-cert\") pod \"e2e-worker-w3-d0a18c-openshift-default-547d57db97-2zhxj\" (UID: \"cf47ace6-4db7-4896-98a4-496dab0e722f\") " pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-2zhxj"
Oct 30 13:14:21.397347 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:14:21.396840    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cf47ace6-4db7-4896-98a4-496dab0e722f-istio-podinfo\") pod \"e2e-worker-w3-d0a18c-openshift-default-547d57db97-2zhxj\" (UID: \"cf47ace6-4db7-4896-98a4-496dab0e722f\") " pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-2zhxj"
Oct 30 13:14:21.397347 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:14:21.396912    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cf47ace6-4db7-4896-98a4-496dab0e722f-istio-data\") pod \"e2e-worker-w3-d0a18c-openshift-default-547d57db97-2zhxj\" (UID: \"cf47ace6-4db7-4896-98a4-496dab0e722f\") " pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-2zhxj"
Oct 30 13:14:21.397347 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:14:21.396952    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf47ace6-4db7-4896-98a4-496dab0e722f-credential-socket\") pod \"e2e-worker-w3-d0a18c-openshift-default-547d57db97-2zhxj\" (UID: \"cf47ace6-4db7-4896-98a4-496dab0e722f\") " pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-2zhxj"
Oct 30 13:14:21.397347 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:14:21.396980    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf47ace6-4db7-4896-98a4-496dab0e722f-workload-socket\") pod \"e2e-worker-w3-d0a18c-openshift-default-547d57db97-2zhxj\" (UID: \"cf47ace6-4db7-4896-98a4-496dab0e722f\") " pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-2zhxj"
Oct 30 13:14:21.397347 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:14:21.397003    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cf47ace6-4db7-4896-98a4-496dab0e722f-workload-certs\") pod \"e2e-worker-w3-d0a18c-openshift-default-547d57db97-2zhxj\" (UID: \"cf47ace6-4db7-4896-98a4-496dab0e722f\") " pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-2zhxj"
Oct 30 13:14:21.397347 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:14:21.397047    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cf47ace6-4db7-4896-98a4-496dab0e722f-istio-envoy\") pod \"e2e-worker-w3-d0a18c-openshift-default-547d57db97-2zhxj\" (UID: \"cf47ace6-4db7-4896-98a4-496dab0e722f\") " pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-2zhxj"
Oct 30 13:14:21.497895 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:14:21.497858    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf47ace6-4db7-4896-98a4-496dab0e722f-workload-socket\") pod \"e2e-worker-w3-d0a18c-openshift-default-547d57db97-2zhxj\" (UID: \"cf47ace6-4db7-4896-98a4-496dab0e722f\") " pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-2zhxj"
Oct 30 13:14:21.497895 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:14:21.497893    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cf47ace6-4db7-4896-98a4-496dab0e722f-workload-certs\") pod \"e2e-worker-w3-d0a18c-openshift-default-547d57db97-2zhxj\" (UID: \"cf47ace6-4db7-4896-98a4-496dab0e722f\") " pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-2zhxj"
Oct 30 13:14:21.498140 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:14:21.497923    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cf47ace6-4db7-4896-98a4-496dab0e722f-istio-envoy\") pod \"e2e-worker-w3-d0a18c-openshift-default-547d57db97-2zhxj\" (UID: \"cf47ace6-4db7-4896-98a4-496dab0e722f\") " pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-2zhxj"
Oct 30 13:14:21.498140 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:14:21.498016    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cf47ace6-4db7-4896-98a4-496dab0e722f-istio-token\") pod \"e2e-worker-w3-d0a18c-openshift-default-547d57db97-2zhxj\" (UID: \"cf47ace6-4db7-4896-98a4-496dab0e722f\") " pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-2zhxj"
Oct 30 13:14:21.498140 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:14:21.498090    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zp2tx\" (UniqueName: \"kubernetes.io/projected/cf47ace6-4db7-4896-98a4-496dab0e722f-kube-api-access-zp2tx\") pod \"e2e-worker-w3-d0a18c-openshift-default-547d57db97-2zhxj\" (UID: \"cf47ace6-4db7-4896-98a4-496dab0e722f\") " pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-2zhxj"
Oct 30 13:14:21.498140 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:14:21.498128    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cf47ace6-4db7-4896-98a4-496dab0e722f-istiod-ca-cert\") pod \"e2e-worker-w3-d0a18c-openshift-default-547d57db97-2zhxj\" (UID: \"cf47ace6-4db7-4896-98a4-496dab0e722f\") " pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-2zhxj"
Oct 30 13:14:21.498399 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:14:21.498185    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cf47ace6-4db7-4896-98a4-496dab0e722f-istio-podinfo\") pod \"e2e-worker-w3-d0a18c-openshift-default-547d57db97-2zhxj\" (UID: \"cf47ace6-4db7-4896-98a4-496dab0e722f\") " pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-2zhxj"
Oct 30 13:14:21.498399 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:14:21.498211    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cf47ace6-4db7-4896-98a4-496dab0e722f-istio-data\") pod \"e2e-worker-w3-d0a18c-openshift-default-547d57db97-2zhxj\" (UID: \"cf47ace6-4db7-4896-98a4-496dab0e722f\") " pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-2zhxj"
Oct 30 13:14:21.498399 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:14:21.498237    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf47ace6-4db7-4896-98a4-496dab0e722f-credential-socket\") pod \"e2e-worker-w3-d0a18c-openshift-default-547d57db97-2zhxj\" (UID: \"cf47ace6-4db7-4896-98a4-496dab0e722f\") " pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-2zhxj"
Oct 30 13:14:21.498399 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:14:21.498309    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf47ace6-4db7-4896-98a4-496dab0e722f-workload-socket\") pod \"e2e-worker-w3-d0a18c-openshift-default-547d57db97-2zhxj\" (UID: \"cf47ace6-4db7-4896-98a4-496dab0e722f\") " pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-2zhxj"
Oct 30 13:14:21.498625 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:14:21.498598    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf47ace6-4db7-4896-98a4-496dab0e722f-credential-socket\") pod \"e2e-worker-w3-d0a18c-openshift-default-547d57db97-2zhxj\" (UID: \"cf47ace6-4db7-4896-98a4-496dab0e722f\") " pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-2zhxj"
Oct 30 13:14:21.498697 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:14:21.498651    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cf47ace6-4db7-4896-98a4-496dab0e722f-istio-data\") pod \"e2e-worker-w3-d0a18c-openshift-default-547d57db97-2zhxj\" (UID: \"cf47ace6-4db7-4896-98a4-496dab0e722f\") " pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-2zhxj"
Oct 30 13:14:21.498918 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:14:21.498892    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cf47ace6-4db7-4896-98a4-496dab0e722f-workload-certs\") pod \"e2e-worker-w3-d0a18c-openshift-default-547d57db97-2zhxj\" (UID: \"cf47ace6-4db7-4896-98a4-496dab0e722f\") " pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-2zhxj"
Oct 30 13:14:21.499061 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:14:21.499028    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cf47ace6-4db7-4896-98a4-496dab0e722f-istiod-ca-cert\") pod \"e2e-worker-w3-d0a18c-openshift-default-547d57db97-2zhxj\" (UID: \"cf47ace6-4db7-4896-98a4-496dab0e722f\") " pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-2zhxj"
Oct 30 13:14:21.500789 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:14:21.500767    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cf47ace6-4db7-4896-98a4-496dab0e722f-istio-envoy\") pod \"e2e-worker-w3-d0a18c-openshift-default-547d57db97-2zhxj\" (UID: \"cf47ace6-4db7-4896-98a4-496dab0e722f\") " pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-2zhxj"
Oct 30 13:14:21.501041 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:14:21.501021    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cf47ace6-4db7-4896-98a4-496dab0e722f-istio-podinfo\") pod \"e2e-worker-w3-d0a18c-openshift-default-547d57db97-2zhxj\" (UID: \"cf47ace6-4db7-4896-98a4-496dab0e722f\") " pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-2zhxj"
Oct 30 13:14:21.524123 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:14:21.524084    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zp2tx\" (UniqueName: \"kubernetes.io/projected/cf47ace6-4db7-4896-98a4-496dab0e722f-kube-api-access-zp2tx\") pod \"e2e-worker-w3-d0a18c-openshift-default-547d57db97-2zhxj\" (UID: \"cf47ace6-4db7-4896-98a4-496dab0e722f\") " pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-2zhxj"
Oct 30 13:14:21.524213 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:14:21.524177    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cf47ace6-4db7-4896-98a4-496dab0e722f-istio-token\") pod \"e2e-worker-w3-d0a18c-openshift-default-547d57db97-2zhxj\" (UID: \"cf47ace6-4db7-4896-98a4-496dab0e722f\") " pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-2zhxj"
Oct 30 13:14:21.598829 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:14:21.598774    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-2zhxj"
Oct 30 13:14:21.678511 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:14:21.678479    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-lfhqw-rcd8z" event={"ID":"1494fd10-5e14-4d43-aeaf-08e3d779b232","Type":"ContainerStarted","Data":"244536df8fce8dcc781a0e7c315b46d6bbed2fdc13ce9e3072bcaf2e5df688eb"}
Oct 30 13:14:21.678640 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:14:21.678532    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-lfhqw-rcd8z" event={"ID":"1494fd10-5e14-4d43-aeaf-08e3d779b232","Type":"ContainerStarted","Data":"a1166ce031876db5c135a72996c45da9e61c360753e7092a66c63e2ee01c62a9"}
Oct 30 13:14:21.773373 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:14:21.773323    4943 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-insights/periodic-gathering-lfhqw-rcd8z" podStartSLOduration=1.7733083729999999 podStartE2EDuration="1.773308373s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:14:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:14:21.772316081 +0000 UTC m=+1037.560319811" watchObservedRunningTime="2026-10-30 13:14:21.773308373 +0000 UTC m=+1037.561312148"
Oct 30 13:14:21.946832 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:14:21.946803    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-2zhxj"]
Oct 30 13:14:21.947389 ip-10-0-142-84 kubenswrapper[4943]: W1030 13:14:21.947364    4943 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcf47ace6_4db7_4896_98a4_496dab0e722f.slice/crio-045c2b4d07ecf16da58ab760f733bfb124480a45354ae0817b4afd6ee2960c69 WatchSource:0}: Error finding container 045c2b4d07ecf16da58ab760f733bfb124480a45354ae0817b4afd6ee2960c69: Status 404 returned error can't find the container with id 045c2b4d07ecf16da58ab760f733bfb124480a45354ae0817b4afd6ee2960c69
Oct 30 13:14:21.949846 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:14:21.949812    4943 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 13:14:21.949927 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:14:21.949873    4943 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 13:14:21.949927 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:14:21.949904    4943 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 13:14:22.298171 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:14:22.298094    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-hdmrm"]
Oct 30 13:14:22.693036 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:14:22.692998    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-2zhxj" event={"ID":"cf47ace6-4db7-4896-98a4-496dab0e722f","Type":"ContainerStarted","Data":"479a3b20dd15fe9271601fddfd828062fcff28532944df485c2ebee2756c0d76"}
Oct 30 13:14:22.693516 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:14:22.693065    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-2zhxj" event={"ID":"cf47ace6-4db7-4896-98a4-496dab0e722f","Type":"ContainerStarted","Data":"045c2b4d07ecf16da58ab760f733bfb124480a45354ae0817b4afd6ee2960c69"}
Oct 30 13:14:22.743711 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:14:22.743656    4943 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-2zhxj" podStartSLOduration=1.743643364 podStartE2EDuration="1.743643364s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:14:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:14:22.74118887 +0000 UTC m=+1038.529192600" watchObservedRunningTime="2026-10-30 13:14:22.743643364 +0000 UTC m=+1038.531647093"
Oct 30 13:14:23.599882 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:14:23.599846    4943 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-2zhxj"
Oct 30 13:14:23.601627 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:14:23.601598    4943 patch_prober.go:29] interesting pod/e2e-worker-w3-d0a18c-openshift-default-547d57db97-2zhxj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused" start-of-body=
Oct 30 13:14:23.601742 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:14:23.601650    4943 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-2zhxj" podUID="cf47ace6-4db7-4896-98a4-496dab0e722f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused"
Oct 30 13:14:24.599580 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:14:24.599543    4943 patch_prober.go:29] interesting pod/e2e-worker-w3-d0a18c-openshift-default-547d57db97-2zhxj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused" start-of-body=
Oct 30 13:14:24.599945 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:14:24.599596    4943 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-2zhxj" podUID="cf47ace6-4db7-4896-98a4-496dab0e722f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused"
Oct 30 13:14:25.599701 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:14:25.599667    4943 patch_prober.go:29] interesting pod/e2e-worker-w3-d0a18c-openshift-default-547d57db97-2zhxj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused" start-of-body=
Oct 30 13:14:25.600101 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:14:25.599720    4943 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-2zhxj" podUID="cf47ace6-4db7-4896-98a4-496dab0e722f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused"
Oct 30 13:14:26.600078 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:14:26.600042    4943 patch_prober.go:29] interesting pod/e2e-worker-w3-d0a18c-openshift-default-547d57db97-2zhxj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused" start-of-body=
Oct 30 13:14:26.600495 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:14:26.600101    4943 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-2zhxj" podUID="cf47ace6-4db7-4896-98a4-496dab0e722f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused"
Oct 30 13:14:27.599499 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:14:27.599459    4943 patch_prober.go:29] interesting pod/e2e-worker-w3-d0a18c-openshift-default-547d57db97-2zhxj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused" start-of-body=
Oct 30 13:14:27.599672 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:14:27.599514    4943 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-2zhxj" podUID="cf47ace6-4db7-4896-98a4-496dab0e722f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused"
Oct 30 13:14:28.603993 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:14:28.603960    4943 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-2zhxj"
Oct 30 13:14:28.604481 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:14:28.604370    4943 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-2zhxj"
Oct 30 13:14:28.605339 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:14:28.605314    4943 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-2zhxj"
Oct 30 13:14:30.625429 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:14:30.625395    4943 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-bf7pz"]
Oct 30 13:14:30.629024 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:14:30.629007    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-bf7pz"
Oct 30 13:14:30.669614 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:14:30.669575    4943 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w2-f76c09-openshift-default-dockercfg-z6xv9\""
Oct 30 13:14:30.672973 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:14:30.672940    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-bf7pz"]
Oct 30 13:14:30.731344 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:14:30.731317    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/258db5e7-aa22-4bb6-b8ee-844241ca5399-istio-data\") pod \"e2e-worker-w2-f76c09-openshift-default-65d5845b67-bf7pz\" (UID: \"258db5e7-aa22-4bb6-b8ee-844241ca5399\") " pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-bf7pz"
Oct 30 13:14:30.731471 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:14:30.731354    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vxwjt\" (UniqueName: \"kubernetes.io/projected/258db5e7-aa22-4bb6-b8ee-844241ca5399-kube-api-access-vxwjt\") pod \"e2e-worker-w2-f76c09-openshift-default-65d5845b67-bf7pz\" (UID: \"258db5e7-aa22-4bb6-b8ee-844241ca5399\") " pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-bf7pz"
Oct 30 13:14:30.731471 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:14:30.731383    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/258db5e7-aa22-4bb6-b8ee-844241ca5399-istio-envoy\") pod \"e2e-worker-w2-f76c09-openshift-default-65d5845b67-bf7pz\" (UID: \"258db5e7-aa22-4bb6-b8ee-844241ca5399\") " pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-bf7pz"
Oct 30 13:14:30.731471 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:14:30.731441    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/258db5e7-aa22-4bb6-b8ee-844241ca5399-workload-certs\") pod \"e2e-worker-w2-f76c09-openshift-default-65d5845b67-bf7pz\" (UID: \"258db5e7-aa22-4bb6-b8ee-844241ca5399\") " pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-bf7pz"
Oct 30 13:14:30.731568 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:14:30.731555    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/258db5e7-aa22-4bb6-b8ee-844241ca5399-workload-socket\") pod \"e2e-worker-w2-f76c09-openshift-default-65d5845b67-bf7pz\" (UID: \"258db5e7-aa22-4bb6-b8ee-844241ca5399\") " pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-bf7pz"
Oct 30 13:14:30.731602 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:14:30.731594    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/258db5e7-aa22-4bb6-b8ee-844241ca5399-istiod-ca-cert\") pod \"e2e-worker-w2-f76c09-openshift-default-65d5845b67-bf7pz\" (UID: \"258db5e7-aa22-4bb6-b8ee-844241ca5399\") " pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-bf7pz"
Oct 30 13:14:30.731638 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:14:30.731618    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/258db5e7-aa22-4bb6-b8ee-844241ca5399-istio-token\") pod \"e2e-worker-w2-f76c09-openshift-default-65d5845b67-bf7pz\" (UID: \"258db5e7-aa22-4bb6-b8ee-844241ca5399\") " pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-bf7pz"
Oct 30 13:14:30.731737 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:14:30.731721    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/258db5e7-aa22-4bb6-b8ee-844241ca5399-credential-socket\") pod \"e2e-worker-w2-f76c09-openshift-default-65d5845b67-bf7pz\" (UID: \"258db5e7-aa22-4bb6-b8ee-844241ca5399\") " pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-bf7pz"
Oct 30 13:14:30.731784 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:14:30.731769    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/258db5e7-aa22-4bb6-b8ee-844241ca5399-istio-podinfo\") pod \"e2e-worker-w2-f76c09-openshift-default-65d5845b67-bf7pz\" (UID: \"258db5e7-aa22-4bb6-b8ee-844241ca5399\") " pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-bf7pz"
Oct 30 13:14:30.832930 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:14:30.832899    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/258db5e7-aa22-4bb6-b8ee-844241ca5399-workload-certs\") pod \"e2e-worker-w2-f76c09-openshift-default-65d5845b67-bf7pz\" (UID: \"258db5e7-aa22-4bb6-b8ee-844241ca5399\") " pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-bf7pz"
Oct 30 13:14:30.833071 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:14:30.832948    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/258db5e7-aa22-4bb6-b8ee-844241ca5399-workload-socket\") pod \"e2e-worker-w2-f76c09-openshift-default-65d5845b67-bf7pz\" (UID: \"258db5e7-aa22-4bb6-b8ee-844241ca5399\") " pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-bf7pz"
Oct 30 13:14:30.833071 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:14:30.832981    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/258db5e7-aa22-4bb6-b8ee-844241ca5399-istiod-ca-cert\") pod \"e2e-worker-w2-f76c09-openshift-default-65d5845b67-bf7pz\" (UID: \"258db5e7-aa22-4bb6-b8ee-844241ca5399\") " pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-bf7pz"
Oct 30 13:14:30.833071 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:14:30.833004    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/258db5e7-aa22-4bb6-b8ee-844241ca5399-istio-token\") pod \"e2e-worker-w2-f76c09-openshift-default-65d5845b67-bf7pz\" (UID: \"258db5e7-aa22-4bb6-b8ee-844241ca5399\") " pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-bf7pz"
Oct 30 13:14:30.833071 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:14:30.833039    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/258db5e7-aa22-4bb6-b8ee-844241ca5399-credential-socket\") pod \"e2e-worker-w2-f76c09-openshift-default-65d5845b67-bf7pz\" (UID: \"258db5e7-aa22-4bb6-b8ee-844241ca5399\") " pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-bf7pz"
Oct 30 13:14:30.833256 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:14:30.833094    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/258db5e7-aa22-4bb6-b8ee-844241ca5399-istio-podinfo\") pod \"e2e-worker-w2-f76c09-openshift-default-65d5845b67-bf7pz\" (UID: \"258db5e7-aa22-4bb6-b8ee-844241ca5399\") " pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-bf7pz"
Oct 30 13:14:30.833256 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:14:30.833145    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/258db5e7-aa22-4bb6-b8ee-844241ca5399-istio-data\") pod \"e2e-worker-w2-f76c09-openshift-default-65d5845b67-bf7pz\" (UID: \"258db5e7-aa22-4bb6-b8ee-844241ca5399\") " pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-bf7pz"
Oct 30 13:14:30.833415 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:14:30.833382    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vxwjt\" (UniqueName: \"kubernetes.io/projected/258db5e7-aa22-4bb6-b8ee-844241ca5399-kube-api-access-vxwjt\") pod \"e2e-worker-w2-f76c09-openshift-default-65d5845b67-bf7pz\" (UID: \"258db5e7-aa22-4bb6-b8ee-844241ca5399\") " pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-bf7pz"
Oct 30 13:14:30.833460 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:14:30.833433    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/258db5e7-aa22-4bb6-b8ee-844241ca5399-workload-socket\") pod \"e2e-worker-w2-f76c09-openshift-default-65d5845b67-bf7pz\" (UID: \"258db5e7-aa22-4bb6-b8ee-844241ca5399\") " pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-bf7pz"
Oct 30 13:14:30.833495 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:14:30.833389    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/258db5e7-aa22-4bb6-b8ee-844241ca5399-workload-certs\") pod \"e2e-worker-w2-f76c09-openshift-default-65d5845b67-bf7pz\" (UID: \"258db5e7-aa22-4bb6-b8ee-844241ca5399\") " pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-bf7pz"
Oct 30 13:14:30.833495 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:14:30.833467    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/258db5e7-aa22-4bb6-b8ee-844241ca5399-istio-envoy\") pod \"e2e-worker-w2-f76c09-openshift-default-65d5845b67-bf7pz\" (UID: \"258db5e7-aa22-4bb6-b8ee-844241ca5399\") " pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-bf7pz"
Oct 30 13:14:30.833664 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:14:30.833643    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/258db5e7-aa22-4bb6-b8ee-844241ca5399-istio-data\") pod \"e2e-worker-w2-f76c09-openshift-default-65d5845b67-bf7pz\" (UID: \"258db5e7-aa22-4bb6-b8ee-844241ca5399\") " pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-bf7pz"
Oct 30 13:14:30.833762 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:14:30.833731    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/258db5e7-aa22-4bb6-b8ee-844241ca5399-credential-socket\") pod \"e2e-worker-w2-f76c09-openshift-default-65d5845b67-bf7pz\" (UID: \"258db5e7-aa22-4bb6-b8ee-844241ca5399\") " pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-bf7pz"
Oct 30 13:14:30.833942 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:14:30.833924    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/258db5e7-aa22-4bb6-b8ee-844241ca5399-istiod-ca-cert\") pod \"e2e-worker-w2-f76c09-openshift-default-65d5845b67-bf7pz\" (UID: \"258db5e7-aa22-4bb6-b8ee-844241ca5399\") " pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-bf7pz"
Oct 30 13:14:30.835725 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:14:30.835699    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/258db5e7-aa22-4bb6-b8ee-844241ca5399-istio-envoy\") pod \"e2e-worker-w2-f76c09-openshift-default-65d5845b67-bf7pz\" (UID: \"258db5e7-aa22-4bb6-b8ee-844241ca5399\") " pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-bf7pz"
Oct 30 13:14:30.836068 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:14:30.836045    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/258db5e7-aa22-4bb6-b8ee-844241ca5399-istio-podinfo\") pod \"e2e-worker-w2-f76c09-openshift-default-65d5845b67-bf7pz\" (UID: \"258db5e7-aa22-4bb6-b8ee-844241ca5399\") " pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-bf7pz"
Oct 30 13:14:30.864290 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:14:30.863372    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vxwjt\" (UniqueName: \"kubernetes.io/projected/258db5e7-aa22-4bb6-b8ee-844241ca5399-kube-api-access-vxwjt\") pod \"e2e-worker-w2-f76c09-openshift-default-65d5845b67-bf7pz\" (UID: \"258db5e7-aa22-4bb6-b8ee-844241ca5399\") " pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-bf7pz"
Oct 30 13:14:30.876255 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:14:30.876145    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/258db5e7-aa22-4bb6-b8ee-844241ca5399-istio-token\") pod \"e2e-worker-w2-f76c09-openshift-default-65d5845b67-bf7pz\" (UID: \"258db5e7-aa22-4bb6-b8ee-844241ca5399\") " pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-bf7pz"
Oct 30 13:14:30.942134 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:14:30.942104    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-bf7pz"
Oct 30 13:14:31.290340 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:14:31.290314    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-bf7pz"]
Oct 30 13:14:31.291525 ip-10-0-142-84 kubenswrapper[4943]: W1030 13:14:31.291502    4943 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod258db5e7_aa22_4bb6_b8ee_844241ca5399.slice/crio-293c778564cc8d05bc32dc0c70ae0ce60b0f8c4357a415370181aefd18320b28 WatchSource:0}: Error finding container 293c778564cc8d05bc32dc0c70ae0ce60b0f8c4357a415370181aefd18320b28: Status 404 returned error can't find the container with id 293c778564cc8d05bc32dc0c70ae0ce60b0f8c4357a415370181aefd18320b28
Oct 30 13:14:31.293788 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:14:31.293758    4943 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 13:14:31.293840 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:14:31.293819    4943 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 13:14:31.293879 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:14:31.293851    4943 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 13:14:31.826072 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:14:31.826033    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-bf7pz" event={"ID":"258db5e7-aa22-4bb6-b8ee-844241ca5399","Type":"ContainerStarted","Data":"9989ec3323990145f3437b91850185cbffab2c42b9e6f132674e969ffb52b858"}
Oct 30 13:14:31.826481 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:14:31.826092    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-bf7pz" event={"ID":"258db5e7-aa22-4bb6-b8ee-844241ca5399","Type":"ContainerStarted","Data":"293c778564cc8d05bc32dc0c70ae0ce60b0f8c4357a415370181aefd18320b28"}
Oct 30 13:14:31.943068 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:14:31.943032    4943 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-bf7pz"
Oct 30 13:14:32.944908 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:14:32.944861    4943 patch_prober.go:29] interesting pod/e2e-worker-w2-f76c09-openshift-default-65d5845b67-bf7pz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused" start-of-body=
Oct 30 13:14:32.945331 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:14:32.944935    4943 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-bf7pz" podUID="258db5e7-aa22-4bb6-b8ee-844241ca5399" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused"
Oct 30 13:14:33.943456 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:14:33.943421    4943 patch_prober.go:29] interesting pod/e2e-worker-w2-f76c09-openshift-default-65d5845b67-bf7pz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused" start-of-body=
Oct 30 13:14:33.943629 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:14:33.943484    4943 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-bf7pz" podUID="258db5e7-aa22-4bb6-b8ee-844241ca5399" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused"
Oct 30 13:14:34.433997 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:14:34.433942    4943 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-bf7pz" podStartSLOduration=4.433929804 podStartE2EDuration="4.433929804s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:14:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:14:31.899965974 +0000 UTC m=+1047.687969704" watchObservedRunningTime="2026-10-30 13:14:34.433929804 +0000 UTC m=+1050.221933533"
Oct 30 13:14:34.436713 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:14:34.436686    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-hdmrm"]
Oct 30 13:14:34.943281 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:14:34.943228    4943 patch_prober.go:29] interesting pod/e2e-worker-w2-f76c09-openshift-default-65d5845b67-bf7pz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused" start-of-body=
Oct 30 13:14:34.943462 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:14:34.943304    4943 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-bf7pz" podUID="258db5e7-aa22-4bb6-b8ee-844241ca5399" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused"
Oct 30 13:14:35.947619 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:14:35.945968    4943 patch_prober.go:29] interesting pod/e2e-worker-w2-f76c09-openshift-default-65d5845b67-bf7pz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused" start-of-body=
Oct 30 13:14:35.947619 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:14:35.946045    4943 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-bf7pz" podUID="258db5e7-aa22-4bb6-b8ee-844241ca5399" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused"
Oct 30 13:14:36.019294 ip-10-0-142-84 kubenswrapper[4943]: E1030 13:14:36.016776    4943 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-knqz6" containerName="extractor"
Oct 30 13:14:36.942582 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:14:36.942548    4943 patch_prober.go:29] interesting pod/e2e-worker-w2-f76c09-openshift-default-65d5845b67-bf7pz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused" start-of-body=
Oct 30 13:14:36.942751 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:14:36.942599    4943 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-bf7pz" podUID="258db5e7-aa22-4bb6-b8ee-844241ca5399" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused"
Oct 30 13:14:37.946984 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:14:37.946951    4943 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-bf7pz"
Oct 30 13:14:37.947534 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:14:37.947513    4943 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-bf7pz"
Oct 30 13:14:37.948289 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:14:37.948250    4943 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-bf7pz"
Oct 30 13:14:46.655455 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:14:46.655416    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-hdmrm"]
Oct 30 13:14:50.611701 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:14:50.611665    4943 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-zmddf"]
Oct 30 13:14:50.614681 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:14:50.614660    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-zmddf"
Oct 30 13:14:50.637476 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:14:50.637451    4943 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w0-031737-openshift-default-dockercfg-77v2d\""
Oct 30 13:14:50.666514 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:14:50.666488    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-zmddf"]
Oct 30 13:14:50.724001 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:14:50.723969    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/78fecfa2-495c-4690-bd0e-08e0a4213ade-istiod-ca-cert\") pod \"e2e-worker-w0-031737-openshift-default-64b8f46f47-zmddf\" (UID: \"78fecfa2-495c-4690-bd0e-08e0a4213ade\") " pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-zmddf"
Oct 30 13:14:50.724136 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:14:50.724005    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-txfhd\" (UniqueName: \"kubernetes.io/projected/78fecfa2-495c-4690-bd0e-08e0a4213ade-kube-api-access-txfhd\") pod \"e2e-worker-w0-031737-openshift-default-64b8f46f47-zmddf\" (UID: \"78fecfa2-495c-4690-bd0e-08e0a4213ade\") " pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-zmddf"
Oct 30 13:14:50.724136 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:14:50.724029    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/78fecfa2-495c-4690-bd0e-08e0a4213ade-istio-envoy\") pod \"e2e-worker-w0-031737-openshift-default-64b8f46f47-zmddf\" (UID: \"78fecfa2-495c-4690-bd0e-08e0a4213ade\") " pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-zmddf"
Oct 30 13:14:50.724249 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:14:50.724225    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/78fecfa2-495c-4690-bd0e-08e0a4213ade-istio-token\") pod \"e2e-worker-w0-031737-openshift-default-64b8f46f47-zmddf\" (UID: \"78fecfa2-495c-4690-bd0e-08e0a4213ade\") " pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-zmddf"
Oct 30 13:14:50.724336 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:14:50.724318    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/78fecfa2-495c-4690-bd0e-08e0a4213ade-istio-podinfo\") pod \"e2e-worker-w0-031737-openshift-default-64b8f46f47-zmddf\" (UID: \"78fecfa2-495c-4690-bd0e-08e0a4213ade\") " pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-zmddf"
Oct 30 13:14:50.724389 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:14:50.724373    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/78fecfa2-495c-4690-bd0e-08e0a4213ade-workload-certs\") pod \"e2e-worker-w0-031737-openshift-default-64b8f46f47-zmddf\" (UID: \"78fecfa2-495c-4690-bd0e-08e0a4213ade\") " pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-zmddf"
Oct 30 13:14:50.724423 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:14:50.724398    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/78fecfa2-495c-4690-bd0e-08e0a4213ade-istio-data\") pod \"e2e-worker-w0-031737-openshift-default-64b8f46f47-zmddf\" (UID: \"78fecfa2-495c-4690-bd0e-08e0a4213ade\") " pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-zmddf"
Oct 30 13:14:50.724461 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:14:50.724441    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/78fecfa2-495c-4690-bd0e-08e0a4213ade-workload-socket\") pod \"e2e-worker-w0-031737-openshift-default-64b8f46f47-zmddf\" (UID: \"78fecfa2-495c-4690-bd0e-08e0a4213ade\") " pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-zmddf"
Oct 30 13:14:50.724493 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:14:50.724474    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/78fecfa2-495c-4690-bd0e-08e0a4213ade-credential-socket\") pod \"e2e-worker-w0-031737-openshift-default-64b8f46f47-zmddf\" (UID: \"78fecfa2-495c-4690-bd0e-08e0a4213ade\") " pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-zmddf"
Oct 30 13:14:50.825154 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:14:50.825125    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/78fecfa2-495c-4690-bd0e-08e0a4213ade-istio-token\") pod \"e2e-worker-w0-031737-openshift-default-64b8f46f47-zmddf\" (UID: \"78fecfa2-495c-4690-bd0e-08e0a4213ade\") " pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-zmddf"
Oct 30 13:14:50.825304 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:14:50.825173    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/78fecfa2-495c-4690-bd0e-08e0a4213ade-istio-podinfo\") pod \"e2e-worker-w0-031737-openshift-default-64b8f46f47-zmddf\" (UID: \"78fecfa2-495c-4690-bd0e-08e0a4213ade\") " pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-zmddf"
Oct 30 13:14:50.825304 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:14:50.825225    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/78fecfa2-495c-4690-bd0e-08e0a4213ade-workload-certs\") pod \"e2e-worker-w0-031737-openshift-default-64b8f46f47-zmddf\" (UID: \"78fecfa2-495c-4690-bd0e-08e0a4213ade\") " pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-zmddf"
Oct 30 13:14:50.825304 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:14:50.825248    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/78fecfa2-495c-4690-bd0e-08e0a4213ade-istio-data\") pod \"e2e-worker-w0-031737-openshift-default-64b8f46f47-zmddf\" (UID: \"78fecfa2-495c-4690-bd0e-08e0a4213ade\") " pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-zmddf"
Oct 30 13:14:50.825451 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:14:50.825316    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/78fecfa2-495c-4690-bd0e-08e0a4213ade-workload-socket\") pod \"e2e-worker-w0-031737-openshift-default-64b8f46f47-zmddf\" (UID: \"78fecfa2-495c-4690-bd0e-08e0a4213ade\") " pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-zmddf"
Oct 30 13:14:50.825451 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:14:50.825340    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/78fecfa2-495c-4690-bd0e-08e0a4213ade-credential-socket\") pod \"e2e-worker-w0-031737-openshift-default-64b8f46f47-zmddf\" (UID: \"78fecfa2-495c-4690-bd0e-08e0a4213ade\") " pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-zmddf"
Oct 30 13:14:50.825451 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:14:50.825380    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/78fecfa2-495c-4690-bd0e-08e0a4213ade-istiod-ca-cert\") pod \"e2e-worker-w0-031737-openshift-default-64b8f46f47-zmddf\" (UID: \"78fecfa2-495c-4690-bd0e-08e0a4213ade\") " pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-zmddf"
Oct 30 13:14:50.825451 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:14:50.825406    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-txfhd\" (UniqueName: \"kubernetes.io/projected/78fecfa2-495c-4690-bd0e-08e0a4213ade-kube-api-access-txfhd\") pod \"e2e-worker-w0-031737-openshift-default-64b8f46f47-zmddf\" (UID: \"78fecfa2-495c-4690-bd0e-08e0a4213ade\") " pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-zmddf"
Oct 30 13:14:50.825451 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:14:50.825434    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/78fecfa2-495c-4690-bd0e-08e0a4213ade-istio-envoy\") pod \"e2e-worker-w0-031737-openshift-default-64b8f46f47-zmddf\" (UID: \"78fecfa2-495c-4690-bd0e-08e0a4213ade\") " pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-zmddf"
Oct 30 13:14:50.825705 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:14:50.825672    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/78fecfa2-495c-4690-bd0e-08e0a4213ade-workload-certs\") pod \"e2e-worker-w0-031737-openshift-default-64b8f46f47-zmddf\" (UID: \"78fecfa2-495c-4690-bd0e-08e0a4213ade\") " pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-zmddf"
Oct 30 13:14:50.825878 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:14:50.825766    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/78fecfa2-495c-4690-bd0e-08e0a4213ade-istio-data\") pod \"e2e-worker-w0-031737-openshift-default-64b8f46f47-zmddf\" (UID: \"78fecfa2-495c-4690-bd0e-08e0a4213ade\") " pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-zmddf"
Oct 30 13:14:50.825878 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:14:50.825860    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/78fecfa2-495c-4690-bd0e-08e0a4213ade-workload-socket\") pod \"e2e-worker-w0-031737-openshift-default-64b8f46f47-zmddf\" (UID: \"78fecfa2-495c-4690-bd0e-08e0a4213ade\") " pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-zmddf"
Oct 30 13:14:50.826003 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:14:50.825922    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/78fecfa2-495c-4690-bd0e-08e0a4213ade-credential-socket\") pod \"e2e-worker-w0-031737-openshift-default-64b8f46f47-zmddf\" (UID: \"78fecfa2-495c-4690-bd0e-08e0a4213ade\") " pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-zmddf"
Oct 30 13:14:50.826229 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:14:50.826201    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/78fecfa2-495c-4690-bd0e-08e0a4213ade-istiod-ca-cert\") pod \"e2e-worker-w0-031737-openshift-default-64b8f46f47-zmddf\" (UID: \"78fecfa2-495c-4690-bd0e-08e0a4213ade\") " pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-zmddf"
Oct 30 13:14:50.827904 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:14:50.827875    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/78fecfa2-495c-4690-bd0e-08e0a4213ade-istio-podinfo\") pod \"e2e-worker-w0-031737-openshift-default-64b8f46f47-zmddf\" (UID: \"78fecfa2-495c-4690-bd0e-08e0a4213ade\") " pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-zmddf"
Oct 30 13:14:50.828046 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:14:50.828026    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/78fecfa2-495c-4690-bd0e-08e0a4213ade-istio-envoy\") pod \"e2e-worker-w0-031737-openshift-default-64b8f46f47-zmddf\" (UID: \"78fecfa2-495c-4690-bd0e-08e0a4213ade\") " pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-zmddf"
Oct 30 13:14:50.854675 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:14:50.854651    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-txfhd\" (UniqueName: \"kubernetes.io/projected/78fecfa2-495c-4690-bd0e-08e0a4213ade-kube-api-access-txfhd\") pod \"e2e-worker-w0-031737-openshift-default-64b8f46f47-zmddf\" (UID: \"78fecfa2-495c-4690-bd0e-08e0a4213ade\") " pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-zmddf"
Oct 30 13:14:50.855038 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:14:50.855020    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/78fecfa2-495c-4690-bd0e-08e0a4213ade-istio-token\") pod \"e2e-worker-w0-031737-openshift-default-64b8f46f47-zmddf\" (UID: \"78fecfa2-495c-4690-bd0e-08e0a4213ade\") " pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-zmddf"
Oct 30 13:14:50.928595 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:14:50.928571    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-zmddf"
Oct 30 13:14:51.248627 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:14:51.248599    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-zmddf"]
Oct 30 13:14:51.249134 ip-10-0-142-84 kubenswrapper[4943]: W1030 13:14:51.249106    4943 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod78fecfa2_495c_4690_bd0e_08e0a4213ade.slice/crio-90e91ee83c9eaa4621b6e398372133cf0bfcb524c148842734d17b19ead9562d WatchSource:0}: Error finding container 90e91ee83c9eaa4621b6e398372133cf0bfcb524c148842734d17b19ead9562d: Status 404 returned error can't find the container with id 90e91ee83c9eaa4621b6e398372133cf0bfcb524c148842734d17b19ead9562d
Oct 30 13:14:51.251640 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:14:51.251608    4943 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 13:14:51.251718 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:14:51.251673    4943 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 13:14:51.251756 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:14:51.251719    4943 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 13:14:51.958782 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:14:51.958747    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-zmddf" event={"ID":"78fecfa2-495c-4690-bd0e-08e0a4213ade","Type":"ContainerStarted","Data":"e119e449a536de7ed6161856fa5ecbae96bccab870c3312223cac6af51b4d68d"}
Oct 30 13:14:51.959165 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:14:51.958804    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-zmddf" event={"ID":"78fecfa2-495c-4690-bd0e-08e0a4213ade","Type":"ContainerStarted","Data":"90e91ee83c9eaa4621b6e398372133cf0bfcb524c148842734d17b19ead9562d"}
Oct 30 13:14:52.059779 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:14:52.059719    4943 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-zmddf" podStartSLOduration=2.059705539 podStartE2EDuration="2.059705539s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:14:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:14:52.045361856 +0000 UTC m=+1067.833365587" watchObservedRunningTime="2026-10-30 13:14:52.059705539 +0000 UTC m=+1067.847709272"
Oct 30 13:14:52.928747 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:14:52.928714    4943 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-zmddf"
Oct 30 13:14:52.930434 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:14:52.930401    4943 patch_prober.go:29] interesting pod/e2e-worker-w0-031737-openshift-default-64b8f46f47-zmddf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused" start-of-body=
Oct 30 13:14:52.930582 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:14:52.930469    4943 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-zmddf" podUID="78fecfa2-495c-4690-bd0e-08e0a4213ade" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused"
Oct 30 13:14:53.928950 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:14:53.928910    4943 patch_prober.go:29] interesting pod/e2e-worker-w0-031737-openshift-default-64b8f46f47-zmddf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused" start-of-body=
Oct 30 13:14:53.929374 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:14:53.928981    4943 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-zmddf" podUID="78fecfa2-495c-4690-bd0e-08e0a4213ade" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused"
Oct 30 13:14:54.929130 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:14:54.929091    4943 patch_prober.go:29] interesting pod/e2e-worker-w0-031737-openshift-default-64b8f46f47-zmddf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused" start-of-body=
Oct 30 13:14:54.929556 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:14:54.929147    4943 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-zmddf" podUID="78fecfa2-495c-4690-bd0e-08e0a4213ade" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused"
Oct 30 13:14:55.934466 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:14:55.934428    4943 patch_prober.go:29] interesting pod/e2e-worker-w0-031737-openshift-default-64b8f46f47-zmddf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused" start-of-body=
Oct 30 13:14:55.935087 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:14:55.935052    4943 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-zmddf" podUID="78fecfa2-495c-4690-bd0e-08e0a4213ade" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused"
Oct 30 13:14:56.929022 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:14:56.928983    4943 patch_prober.go:29] interesting pod/e2e-worker-w0-031737-openshift-default-64b8f46f47-zmddf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused" start-of-body=
Oct 30 13:14:56.929346 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:14:56.929313    4943 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-zmddf" podUID="78fecfa2-495c-4690-bd0e-08e0a4213ade" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused"
Oct 30 13:14:58.135686 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:14:58.135649    4943 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-zmddf"
Oct 30 13:14:58.136652 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:14:58.136526    4943 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-zmddf"
Oct 30 13:14:58.137314 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:14:58.137292    4943 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-zmddf"
Oct 30 13:15:00.243466 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:00.243430    4943 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-05942d-29889435-gvqj5"]
Oct 30 13:15:00.250803 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:00.250770    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-05942d-29889435-gvqj5"
Oct 30 13:15:00.251610 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:00.251584    4943 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-ab3176-29889435-hnmkz"]
Oct 30 13:15:00.255178 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:00.255157    4943 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-d0a18c-29889435-ds45l"]
Oct 30 13:15:00.255345 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:00.255324    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-ab3176-29889435-hnmkz"
Oct 30 13:15:00.258366 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:00.258345    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-d0a18c-29889435-ds45l"
Oct 30 13:15:00.275699 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:00.275673    4943 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-p9z57\""
Oct 30 13:15:00.279714 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:00.279679    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-ab3176-29889435-hnmkz"]
Oct 30 13:15:00.298290 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:00.295399    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-05942d-29889435-gvqj5"]
Oct 30 13:15:00.302285 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:00.300030    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-d0a18c-29889435-ds45l"]
Oct 30 13:15:00.319835 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:00.319806    4943 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29889435-75l6z"]
Oct 30 13:15:00.324197 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:00.324174    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889435-75l6z"
Oct 30 13:15:00.373105 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:00.367612    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dk48v\" (UniqueName: \"kubernetes.io/projected/fdaf8439-65b5-4e9a-b519-83d40aaf7307-kube-api-access-dk48v\") pod \"maas-api-key-cleanup-e2e-worker-w1-05942d-29889435-gvqj5\" (UID: \"fdaf8439-65b5-4e9a-b519-83d40aaf7307\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-05942d-29889435-gvqj5"
Oct 30 13:15:00.373105 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:00.367704    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lhfph\" (UniqueName: \"kubernetes.io/projected/6719a2ae-c55d-477d-8853-f032599f9d45-kube-api-access-lhfph\") pod \"maas-api-key-cleanup-e2e-worker-w3-d0a18c-29889435-ds45l\" (UID: \"6719a2ae-c55d-477d-8853-f032599f9d45\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-d0a18c-29889435-ds45l"
Oct 30 13:15:00.373105 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:00.367745    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fvm5x\" (UniqueName: \"kubernetes.io/projected/bd5de1ce-17cd-4e87-bbac-766e14841a08-kube-api-access-fvm5x\") pod \"maas-api-key-cleanup-29889435-75l6z\" (UID: \"bd5de1ce-17cd-4e87-bbac-766e14841a08\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889435-75l6z"
Oct 30 13:15:00.373105 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:00.367792    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kgt9d\" (UniqueName: \"kubernetes.io/projected/8ec3dc68-1993-4289-bf82-f4e629c4d7ce-kube-api-access-kgt9d\") pod \"maas-api-key-cleanup-e2e-worker-w6-ab3176-29889435-hnmkz\" (UID: \"8ec3dc68-1993-4289-bf82-f4e629c4d7ce\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-ab3176-29889435-hnmkz"
Oct 30 13:15:00.373105 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:00.367840    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bd5de1ce-17cd-4e87-bbac-766e14841a08-maas-api-service-ca\") pod \"maas-api-key-cleanup-29889435-75l6z\" (UID: \"bd5de1ce-17cd-4e87-bbac-766e14841a08\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889435-75l6z"
Oct 30 13:15:00.373105 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:00.367943    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6719a2ae-c55d-477d-8853-f032599f9d45-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w3-d0a18c-29889435-ds45l\" (UID: \"6719a2ae-c55d-477d-8853-f032599f9d45\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-d0a18c-29889435-ds45l"
Oct 30 13:15:00.373105 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:00.367969    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8ec3dc68-1993-4289-bf82-f4e629c4d7ce-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w6-ab3176-29889435-hnmkz\" (UID: \"8ec3dc68-1993-4289-bf82-f4e629c4d7ce\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-ab3176-29889435-hnmkz"
Oct 30 13:15:00.373105 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:00.368021    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fdaf8439-65b5-4e9a-b519-83d40aaf7307-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w1-05942d-29889435-gvqj5\" (UID: \"fdaf8439-65b5-4e9a-b519-83d40aaf7307\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-05942d-29889435-gvqj5"
Oct 30 13:15:00.373105 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:00.372988    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29889435-75l6z"]
Oct 30 13:15:00.468742 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:00.468706    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bd5de1ce-17cd-4e87-bbac-766e14841a08-maas-api-service-ca\") pod \"maas-api-key-cleanup-29889435-75l6z\" (UID: \"bd5de1ce-17cd-4e87-bbac-766e14841a08\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889435-75l6z"
Oct 30 13:15:00.468920 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:00.468799    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6719a2ae-c55d-477d-8853-f032599f9d45-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w3-d0a18c-29889435-ds45l\" (UID: \"6719a2ae-c55d-477d-8853-f032599f9d45\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-d0a18c-29889435-ds45l"
Oct 30 13:15:00.468920 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:00.468819    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8ec3dc68-1993-4289-bf82-f4e629c4d7ce-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w6-ab3176-29889435-hnmkz\" (UID: \"8ec3dc68-1993-4289-bf82-f4e629c4d7ce\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-ab3176-29889435-hnmkz"
Oct 30 13:15:00.468920 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:00.468848    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fdaf8439-65b5-4e9a-b519-83d40aaf7307-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w1-05942d-29889435-gvqj5\" (UID: \"fdaf8439-65b5-4e9a-b519-83d40aaf7307\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-05942d-29889435-gvqj5"
Oct 30 13:15:00.468920 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:00.468877    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dk48v\" (UniqueName: \"kubernetes.io/projected/fdaf8439-65b5-4e9a-b519-83d40aaf7307-kube-api-access-dk48v\") pod \"maas-api-key-cleanup-e2e-worker-w1-05942d-29889435-gvqj5\" (UID: \"fdaf8439-65b5-4e9a-b519-83d40aaf7307\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-05942d-29889435-gvqj5"
Oct 30 13:15:00.469153 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:00.468938    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lhfph\" (UniqueName: \"kubernetes.io/projected/6719a2ae-c55d-477d-8853-f032599f9d45-kube-api-access-lhfph\") pod \"maas-api-key-cleanup-e2e-worker-w3-d0a18c-29889435-ds45l\" (UID: \"6719a2ae-c55d-477d-8853-f032599f9d45\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-d0a18c-29889435-ds45l"
Oct 30 13:15:00.469153 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:00.468959    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fvm5x\" (UniqueName: \"kubernetes.io/projected/bd5de1ce-17cd-4e87-bbac-766e14841a08-kube-api-access-fvm5x\") pod \"maas-api-key-cleanup-29889435-75l6z\" (UID: \"bd5de1ce-17cd-4e87-bbac-766e14841a08\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889435-75l6z"
Oct 30 13:15:00.469153 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:00.468998    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kgt9d\" (UniqueName: \"kubernetes.io/projected/8ec3dc68-1993-4289-bf82-f4e629c4d7ce-kube-api-access-kgt9d\") pod \"maas-api-key-cleanup-e2e-worker-w6-ab3176-29889435-hnmkz\" (UID: \"8ec3dc68-1993-4289-bf82-f4e629c4d7ce\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-ab3176-29889435-hnmkz"
Oct 30 13:15:00.469572 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:00.469545    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8ec3dc68-1993-4289-bf82-f4e629c4d7ce-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w6-ab3176-29889435-hnmkz\" (UID: \"8ec3dc68-1993-4289-bf82-f4e629c4d7ce\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-ab3176-29889435-hnmkz"
Oct 30 13:15:00.469678 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:00.469659    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bd5de1ce-17cd-4e87-bbac-766e14841a08-maas-api-service-ca\") pod \"maas-api-key-cleanup-29889435-75l6z\" (UID: \"bd5de1ce-17cd-4e87-bbac-766e14841a08\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889435-75l6z"
Oct 30 13:15:00.469740 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:00.469695    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fdaf8439-65b5-4e9a-b519-83d40aaf7307-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w1-05942d-29889435-gvqj5\" (UID: \"fdaf8439-65b5-4e9a-b519-83d40aaf7307\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-05942d-29889435-gvqj5"
Oct 30 13:15:00.469740 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:00.469704    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6719a2ae-c55d-477d-8853-f032599f9d45-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w3-d0a18c-29889435-ds45l\" (UID: \"6719a2ae-c55d-477d-8853-f032599f9d45\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-d0a18c-29889435-ds45l"
Oct 30 13:15:00.499305 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:00.499240    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dk48v\" (UniqueName: \"kubernetes.io/projected/fdaf8439-65b5-4e9a-b519-83d40aaf7307-kube-api-access-dk48v\") pod \"maas-api-key-cleanup-e2e-worker-w1-05942d-29889435-gvqj5\" (UID: \"fdaf8439-65b5-4e9a-b519-83d40aaf7307\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-05942d-29889435-gvqj5"
Oct 30 13:15:00.508002 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:00.507974    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lhfph\" (UniqueName: \"kubernetes.io/projected/6719a2ae-c55d-477d-8853-f032599f9d45-kube-api-access-lhfph\") pod \"maas-api-key-cleanup-e2e-worker-w3-d0a18c-29889435-ds45l\" (UID: \"6719a2ae-c55d-477d-8853-f032599f9d45\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-d0a18c-29889435-ds45l"
Oct 30 13:15:00.508127 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:00.508080    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fvm5x\" (UniqueName: \"kubernetes.io/projected/bd5de1ce-17cd-4e87-bbac-766e14841a08-kube-api-access-fvm5x\") pod \"maas-api-key-cleanup-29889435-75l6z\" (UID: \"bd5de1ce-17cd-4e87-bbac-766e14841a08\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889435-75l6z"
Oct 30 13:15:00.511160 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:00.511138    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kgt9d\" (UniqueName: \"kubernetes.io/projected/8ec3dc68-1993-4289-bf82-f4e629c4d7ce-kube-api-access-kgt9d\") pod \"maas-api-key-cleanup-e2e-worker-w6-ab3176-29889435-hnmkz\" (UID: \"8ec3dc68-1993-4289-bf82-f4e629c4d7ce\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-ab3176-29889435-hnmkz"
Oct 30 13:15:00.566620 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:00.566597    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-05942d-29889435-gvqj5"
Oct 30 13:15:00.574325 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:00.574303    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-ab3176-29889435-hnmkz"
Oct 30 13:15:00.582639 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:00.582613    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-d0a18c-29889435-ds45l"
Oct 30 13:15:00.642094 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:00.642059    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889435-75l6z"
Oct 30 13:15:00.898640 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:00.898606    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-05942d-29889435-gvqj5"]
Oct 30 13:15:00.899190 ip-10-0-142-84 kubenswrapper[4943]: W1030 13:15:00.899160    4943 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfdaf8439_65b5_4e9a_b519_83d40aaf7307.slice/crio-9c6b7303218d632c3c863b190193671a672f110ce029e68a76b68ac2852b80ca WatchSource:0}: Error finding container 9c6b7303218d632c3c863b190193671a672f110ce029e68a76b68ac2852b80ca: Status 404 returned error can't find the container with id 9c6b7303218d632c3c863b190193671a672f110ce029e68a76b68ac2852b80ca
Oct 30 13:15:00.926915 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:00.926887    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-ab3176-29889435-hnmkz"]
Oct 30 13:15:00.931231 ip-10-0-142-84 kubenswrapper[4943]: W1030 13:15:00.931189    4943 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8ec3dc68_1993_4289_bf82_f4e629c4d7ce.slice/crio-424695806e534b4dbccf35f56629d72a7c9cd84fbe1ae31cd7e9f4a83f20c78e WatchSource:0}: Error finding container 424695806e534b4dbccf35f56629d72a7c9cd84fbe1ae31cd7e9f4a83f20c78e: Status 404 returned error can't find the container with id 424695806e534b4dbccf35f56629d72a7c9cd84fbe1ae31cd7e9f4a83f20c78e
Oct 30 13:15:00.957594 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:00.957568    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-d0a18c-29889435-ds45l"]
Oct 30 13:15:00.957997 ip-10-0-142-84 kubenswrapper[4943]: W1030 13:15:00.957974    4943 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6719a2ae_c55d_477d_8853_f032599f9d45.slice/crio-47623692b4a164e330c6f7357f7b18c9496ad8bf06dc059ff37642b9d5cd690b WatchSource:0}: Error finding container 47623692b4a164e330c6f7357f7b18c9496ad8bf06dc059ff37642b9d5cd690b: Status 404 returned error can't find the container with id 47623692b4a164e330c6f7357f7b18c9496ad8bf06dc059ff37642b9d5cd690b
Oct 30 13:15:01.012637 ip-10-0-142-84 kubenswrapper[4943]: W1030 13:15:01.012614    4943 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbd5de1ce_17cd_4e87_bbac_766e14841a08.slice/crio-39b834608cea5321f9120793febb86e7d7cbe31871a70ad8bba0b1e367ef3917 WatchSource:0}: Error finding container 39b834608cea5321f9120793febb86e7d7cbe31871a70ad8bba0b1e367ef3917: Status 404 returned error can't find the container with id 39b834608cea5321f9120793febb86e7d7cbe31871a70ad8bba0b1e367ef3917
Oct 30 13:15:01.012748 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:01.012661    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29889435-75l6z"]
Oct 30 13:15:01.099113 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:01.099071    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-05942d-29889435-gvqj5" event={"ID":"fdaf8439-65b5-4e9a-b519-83d40aaf7307","Type":"ContainerStarted","Data":"9c6b7303218d632c3c863b190193671a672f110ce029e68a76b68ac2852b80ca"}
Oct 30 13:15:01.100300 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:01.100273    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889435-75l6z" event={"ID":"bd5de1ce-17cd-4e87-bbac-766e14841a08","Type":"ContainerStarted","Data":"39b834608cea5321f9120793febb86e7d7cbe31871a70ad8bba0b1e367ef3917"}
Oct 30 13:15:01.101441 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:01.101420    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-ab3176-29889435-hnmkz" event={"ID":"8ec3dc68-1993-4289-bf82-f4e629c4d7ce","Type":"ContainerStarted","Data":"424695806e534b4dbccf35f56629d72a7c9cd84fbe1ae31cd7e9f4a83f20c78e"}
Oct 30 13:15:01.102612 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:01.102591    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-d0a18c-29889435-ds45l" event={"ID":"6719a2ae-c55d-477d-8853-f032599f9d45","Type":"ContainerStarted","Data":"47623692b4a164e330c6f7357f7b18c9496ad8bf06dc059ff37642b9d5cd690b"}
Oct 30 13:15:02.830185 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:02.830141    4943 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-2e9a65a0-42gwd"]
Oct 30 13:15:02.834661 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:02.834631    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-2e9a65a0-42gwd"
Oct 30 13:15:02.860123 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:02.860088    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-2e9a65a0-42gwd"]
Oct 30 13:15:02.868457 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:02.868424    4943 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 13:15:02.932203 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:02.932167    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cx82j\" (UniqueName: \"kubernetes.io/projected/82ef0ce2-e735-491c-8489-509d7aeb82d4-kube-api-access-cx82j\") pod \"maas-api-revoke-sub-2e9a65a0-42gwd\" (UID: \"82ef0ce2-e735-491c-8489-509d7aeb82d4\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-2e9a65a0-42gwd"
Oct 30 13:15:02.932394 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:02.932238    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/82ef0ce2-e735-491c-8489-509d7aeb82d4-maas-api-xks-ca\") pod \"maas-api-revoke-sub-2e9a65a0-42gwd\" (UID: \"82ef0ce2-e735-491c-8489-509d7aeb82d4\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-2e9a65a0-42gwd"
Oct 30 13:15:02.932394 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:02.932347    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/82ef0ce2-e735-491c-8489-509d7aeb82d4-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-2e9a65a0-42gwd\" (UID: \"82ef0ce2-e735-491c-8489-509d7aeb82d4\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-2e9a65a0-42gwd"
Oct 30 13:15:03.033916 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:03.033874    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cx82j\" (UniqueName: \"kubernetes.io/projected/82ef0ce2-e735-491c-8489-509d7aeb82d4-kube-api-access-cx82j\") pod \"maas-api-revoke-sub-2e9a65a0-42gwd\" (UID: \"82ef0ce2-e735-491c-8489-509d7aeb82d4\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-2e9a65a0-42gwd"
Oct 30 13:15:03.034099 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:03.033958    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/82ef0ce2-e735-491c-8489-509d7aeb82d4-maas-api-xks-ca\") pod \"maas-api-revoke-sub-2e9a65a0-42gwd\" (UID: \"82ef0ce2-e735-491c-8489-509d7aeb82d4\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-2e9a65a0-42gwd"
Oct 30 13:15:03.034099 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:03.034044    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/82ef0ce2-e735-491c-8489-509d7aeb82d4-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-2e9a65a0-42gwd\" (UID: \"82ef0ce2-e735-491c-8489-509d7aeb82d4\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-2e9a65a0-42gwd"
Oct 30 13:15:03.034920 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:03.034854    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/82ef0ce2-e735-491c-8489-509d7aeb82d4-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-2e9a65a0-42gwd\" (UID: \"82ef0ce2-e735-491c-8489-509d7aeb82d4\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-2e9a65a0-42gwd"
Oct 30 13:15:03.039095 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:03.039066    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/82ef0ce2-e735-491c-8489-509d7aeb82d4-maas-api-xks-ca\") pod \"maas-api-revoke-sub-2e9a65a0-42gwd\" (UID: \"82ef0ce2-e735-491c-8489-509d7aeb82d4\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-2e9a65a0-42gwd"
Oct 30 13:15:03.062482 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:03.062457    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cx82j\" (UniqueName: \"kubernetes.io/projected/82ef0ce2-e735-491c-8489-509d7aeb82d4-kube-api-access-cx82j\") pod \"maas-api-revoke-sub-2e9a65a0-42gwd\" (UID: \"82ef0ce2-e735-491c-8489-509d7aeb82d4\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-2e9a65a0-42gwd"
Oct 30 13:15:03.153115 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:03.153036    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-2e9a65a0-42gwd"
Oct 30 13:15:03.543143 ip-10-0-142-84 kubenswrapper[4943]: W1030 13:15:03.543099    4943 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod82ef0ce2_e735_491c_8489_509d7aeb82d4.slice/crio-11ff7d0fee3e1aeebf5c068b00c9b8bf4486a3170488688d8e04c16a7960d347 WatchSource:0}: Error finding container 11ff7d0fee3e1aeebf5c068b00c9b8bf4486a3170488688d8e04c16a7960d347: Status 404 returned error can't find the container with id 11ff7d0fee3e1aeebf5c068b00c9b8bf4486a3170488688d8e04c16a7960d347
Oct 30 13:15:03.543407 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:03.543380    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-2e9a65a0-42gwd"]
Oct 30 13:15:04.159035 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:04.158991    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-2e9a65a0-42gwd" event={"ID":"82ef0ce2-e735-491c-8489-509d7aeb82d4","Type":"ContainerStarted","Data":"11ff7d0fee3e1aeebf5c068b00c9b8bf4486a3170488688d8e04c16a7960d347"}
Oct 30 13:15:05.168499 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:05.168460    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-ab3176-29889435-hnmkz" event={"ID":"8ec3dc68-1993-4289-bf82-f4e629c4d7ce","Type":"ContainerStarted","Data":"d79e32b8003cac93a08274d20a3c7540b167db43e1dfdbe07d4acb8d59bd4940"}
Oct 30 13:15:05.170232 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:05.170200    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-05942d-29889435-gvqj5" event={"ID":"fdaf8439-65b5-4e9a-b519-83d40aaf7307","Type":"ContainerStarted","Data":"ec0bf8bd13f691e080b75ae29eda9195a18b18a951c80e31cb0426e537bf5859"}
Oct 30 13:15:05.227475 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:05.227425    4943 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-ab3176-29889435-hnmkz" podStartSLOduration=1.447920589 podStartE2EDuration="5.227411996s" totalImagesPullingTime="3.779491407s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:15:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:15:05.224723898 +0000 UTC m=+1081.012727631" watchObservedRunningTime="2026-10-30 13:15:05.227411996 +0000 UTC m=+1081.015415726"
Oct 30 13:15:05.289479 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:05.289435    4943 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-05942d-29889435-gvqj5" podStartSLOduration=1.860711104 podStartE2EDuration="5.289420979s" totalImagesPullingTime="3.428709875s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:15:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:15:05.285912008 +0000 UTC m=+1081.073915739" watchObservedRunningTime="2026-10-30 13:15:05.289420979 +0000 UTC m=+1081.077424708"
Oct 30 13:15:05.954961 ip-10-0-142-84 kubenswrapper[4943]: E1030 13:15:05.954920    4943 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-knqz6" containerName="extractor"
Oct 30 13:15:06.177573 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:06.177537    4943 generic.go:356] "Generic (PLEG): container finished" podID="bd5de1ce-17cd-4e87-bbac-766e14841a08" containerID="02fdef6a2da1e4a9468fe682759ee852c0f30e5782ebd0fea8d7f54bb7b026f3" exitCode=60
Oct 30 13:15:06.177981 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:06.177621    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889435-75l6z" event={"ID":"bd5de1ce-17cd-4e87-bbac-766e14841a08","Type":"ContainerDied","Data":"02fdef6a2da1e4a9468fe682759ee852c0f30e5782ebd0fea8d7f54bb7b026f3"}
Oct 30 13:15:06.177981 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:06.177841    4943 scope.go:122] "RemoveContainer" containerID="02fdef6a2da1e4a9468fe682759ee852c0f30e5782ebd0fea8d7f54bb7b026f3"
Oct 30 13:15:06.179358 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:06.179308    4943 generic.go:356] "Generic (PLEG): container finished" podID="8ec3dc68-1993-4289-bf82-f4e629c4d7ce" containerID="d79e32b8003cac93a08274d20a3c7540b167db43e1dfdbe07d4acb8d59bd4940" exitCode=60
Oct 30 13:15:06.179415 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:06.179385    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-ab3176-29889435-hnmkz" event={"ID":"8ec3dc68-1993-4289-bf82-f4e629c4d7ce","Type":"ContainerDied","Data":"d79e32b8003cac93a08274d20a3c7540b167db43e1dfdbe07d4acb8d59bd4940"}
Oct 30 13:15:06.179612 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:06.179598    4943 scope.go:122] "RemoveContainer" containerID="d79e32b8003cac93a08274d20a3c7540b167db43e1dfdbe07d4acb8d59bd4940"
Oct 30 13:15:06.181184 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:06.181156    4943 generic.go:356] "Generic (PLEG): container finished" podID="6719a2ae-c55d-477d-8853-f032599f9d45" containerID="1b1bd591518e11641d3c5741cedfcf26aa2db4ee80d36d9ce77b86f03b25efc7" exitCode=60
Oct 30 13:15:06.181327 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:06.181245    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-d0a18c-29889435-ds45l" event={"ID":"6719a2ae-c55d-477d-8853-f032599f9d45","Type":"ContainerDied","Data":"1b1bd591518e11641d3c5741cedfcf26aa2db4ee80d36d9ce77b86f03b25efc7"}
Oct 30 13:15:06.181497 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:06.181472    4943 scope.go:122] "RemoveContainer" containerID="1b1bd591518e11641d3c5741cedfcf26aa2db4ee80d36d9ce77b86f03b25efc7"
Oct 30 13:15:06.183018 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:06.182996    4943 generic.go:356] "Generic (PLEG): container finished" podID="fdaf8439-65b5-4e9a-b519-83d40aaf7307" containerID="ec0bf8bd13f691e080b75ae29eda9195a18b18a951c80e31cb0426e537bf5859" exitCode=60
Oct 30 13:15:06.183122 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:06.183024    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-05942d-29889435-gvqj5" event={"ID":"fdaf8439-65b5-4e9a-b519-83d40aaf7307","Type":"ContainerDied","Data":"ec0bf8bd13f691e080b75ae29eda9195a18b18a951c80e31cb0426e537bf5859"}
Oct 30 13:15:06.183244 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:06.183229    4943 scope.go:122] "RemoveContainer" containerID="ec0bf8bd13f691e080b75ae29eda9195a18b18a951c80e31cb0426e537bf5859"
Oct 30 13:15:06.184606 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:06.184584    4943 generic.go:356] "Generic (PLEG): container finished" podID="82ef0ce2-e735-491c-8489-509d7aeb82d4" containerID="fa5db13b4de531d7eedb9007c9a86fac2d62d0ea1210d17f699bb2e10fc7aa60" exitCode=0
Oct 30 13:15:06.184702 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:06.184614    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-2e9a65a0-42gwd" event={"ID":"82ef0ce2-e735-491c-8489-509d7aeb82d4","Type":"ContainerDied","Data":"fa5db13b4de531d7eedb9007c9a86fac2d62d0ea1210d17f699bb2e10fc7aa60"}
Oct 30 13:15:06.819333 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:06.819226    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-hdmrm"]
Oct 30 13:15:07.194527 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:07.194499    4943 generic.go:356] "Generic (PLEG): container finished" podID="fdaf8439-65b5-4e9a-b519-83d40aaf7307" containerID="b1a770a6e44dcf26ca00852f9b771c93a17373265128e074e8ab4c8147cbd6e4" exitCode=60
Oct 30 13:15:07.194935 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:07.194589    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-05942d-29889435-gvqj5" event={"ID":"fdaf8439-65b5-4e9a-b519-83d40aaf7307","Type":"ContainerDied","Data":"b1a770a6e44dcf26ca00852f9b771c93a17373265128e074e8ab4c8147cbd6e4"}
Oct 30 13:15:07.194935 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:07.194651    4943 scope.go:122] "RemoveContainer" containerID="ec0bf8bd13f691e080b75ae29eda9195a18b18a951c80e31cb0426e537bf5859"
Oct 30 13:15:07.194935 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:07.194923    4943 scope.go:122] "RemoveContainer" containerID="b1a770a6e44dcf26ca00852f9b771c93a17373265128e074e8ab4c8147cbd6e4"
Oct 30 13:15:07.195200 ip-10-0-142-84 kubenswrapper[4943]: E1030 13:15:07.195176    4943 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w1-05942d-29889435-gvqj5_odh-ai-gateway-infra(fdaf8439-65b5-4e9a-b519-83d40aaf7307)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-05942d-29889435-gvqj5" podUID="fdaf8439-65b5-4e9a-b519-83d40aaf7307"
Oct 30 13:15:07.197240 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:07.197219    4943 generic.go:356] "Generic (PLEG): container finished" podID="bd5de1ce-17cd-4e87-bbac-766e14841a08" containerID="cd6cdcc885b81200fdb41dc892ee2500cac4d443520f804cb2ebbf9dbc204320" exitCode=60
Oct 30 13:15:07.197359 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:07.197288    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889435-75l6z" event={"ID":"bd5de1ce-17cd-4e87-bbac-766e14841a08","Type":"ContainerDied","Data":"cd6cdcc885b81200fdb41dc892ee2500cac4d443520f804cb2ebbf9dbc204320"}
Oct 30 13:15:07.197577 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:07.197560    4943 scope.go:122] "RemoveContainer" containerID="cd6cdcc885b81200fdb41dc892ee2500cac4d443520f804cb2ebbf9dbc204320"
Oct 30 13:15:07.197839 ip-10-0-142-84 kubenswrapper[4943]: E1030 13:15:07.197798    4943 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-29889435-75l6z_odh-ai-gateway-infra(bd5de1ce-17cd-4e87-bbac-766e14841a08)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889435-75l6z" podUID="bd5de1ce-17cd-4e87-bbac-766e14841a08"
Oct 30 13:15:07.199372 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:07.199348    4943 generic.go:356] "Generic (PLEG): container finished" podID="8ec3dc68-1993-4289-bf82-f4e629c4d7ce" containerID="25223c0d82cc58dbf03dba699441936c3be600a550c013f29e54564d25eccddf" exitCode=60
Oct 30 13:15:07.199480 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:07.199436    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-ab3176-29889435-hnmkz" event={"ID":"8ec3dc68-1993-4289-bf82-f4e629c4d7ce","Type":"ContainerDied","Data":"25223c0d82cc58dbf03dba699441936c3be600a550c013f29e54564d25eccddf"}
Oct 30 13:15:07.199639 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:07.199622    4943 scope.go:122] "RemoveContainer" containerID="25223c0d82cc58dbf03dba699441936c3be600a550c013f29e54564d25eccddf"
Oct 30 13:15:07.199879 ip-10-0-142-84 kubenswrapper[4943]: E1030 13:15:07.199853    4943 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w6-ab3176-29889435-hnmkz_odh-ai-gateway-infra(8ec3dc68-1993-4289-bf82-f4e629c4d7ce)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-ab3176-29889435-hnmkz" podUID="8ec3dc68-1993-4289-bf82-f4e629c4d7ce"
Oct 30 13:15:07.201662 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:07.201638    4943 generic.go:356] "Generic (PLEG): container finished" podID="6719a2ae-c55d-477d-8853-f032599f9d45" containerID="9a09dfd59262a24e1d3e717d3522862f7e968b3bdb4ae05a749f9d0ca425cd31" exitCode=60
Oct 30 13:15:07.201753 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:07.201669    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-d0a18c-29889435-ds45l" event={"ID":"6719a2ae-c55d-477d-8853-f032599f9d45","Type":"ContainerDied","Data":"9a09dfd59262a24e1d3e717d3522862f7e968b3bdb4ae05a749f9d0ca425cd31"}
Oct 30 13:15:07.201899 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:07.201881    4943 scope.go:122] "RemoveContainer" containerID="9a09dfd59262a24e1d3e717d3522862f7e968b3bdb4ae05a749f9d0ca425cd31"
Oct 30 13:15:07.202124 ip-10-0-142-84 kubenswrapper[4943]: E1030 13:15:07.202103    4943 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w3-d0a18c-29889435-ds45l_odh-ai-gateway-infra(6719a2ae-c55d-477d-8853-f032599f9d45)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-d0a18c-29889435-ds45l" podUID="6719a2ae-c55d-477d-8853-f032599f9d45"
Oct 30 13:15:07.401053 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:07.401031    4943 scope.go:122] "RemoveContainer" containerID="02fdef6a2da1e4a9468fe682759ee852c0f30e5782ebd0fea8d7f54bb7b026f3"
Oct 30 13:15:07.404212 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:07.404188    4943 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-2e9a65a0-42gwd"
Oct 30 13:15:07.415853 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:07.415797    4943 scope.go:122] "RemoveContainer" containerID="d79e32b8003cac93a08274d20a3c7540b167db43e1dfdbe07d4acb8d59bd4940"
Oct 30 13:15:07.430780 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:07.430751    4943 scope.go:122] "RemoveContainer" containerID="1b1bd591518e11641d3c5741cedfcf26aa2db4ee80d36d9ce77b86f03b25efc7"
Oct 30 13:15:07.518283 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:07.497948    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/82ef0ce2-e735-491c-8489-509d7aeb82d4-kube-api-access-cx82j\" (UniqueName: \"kubernetes.io/projected/82ef0ce2-e735-491c-8489-509d7aeb82d4-kube-api-access-cx82j\") pod \"82ef0ce2-e735-491c-8489-509d7aeb82d4\" (UID: \"82ef0ce2-e735-491c-8489-509d7aeb82d4\") "
Oct 30 13:15:07.518283 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:07.498056    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/82ef0ce2-e735-491c-8489-509d7aeb82d4-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/82ef0ce2-e735-491c-8489-509d7aeb82d4-maas-api-xks-ca\") pod \"82ef0ce2-e735-491c-8489-509d7aeb82d4\" (UID: \"82ef0ce2-e735-491c-8489-509d7aeb82d4\") "
Oct 30 13:15:07.518283 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:07.498256    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/82ef0ce2-e735-491c-8489-509d7aeb82d4-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/82ef0ce2-e735-491c-8489-509d7aeb82d4-maas-api-openshift-ca\") pod \"82ef0ce2-e735-491c-8489-509d7aeb82d4\" (UID: \"82ef0ce2-e735-491c-8489-509d7aeb82d4\") "
Oct 30 13:15:07.518283 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:07.509375    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/82ef0ce2-e735-491c-8489-509d7aeb82d4-maas-api-xks-ca" pod "82ef0ce2-e735-491c-8489-509d7aeb82d4" (UID: "82ef0ce2-e735-491c-8489-509d7aeb82d4"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 13:15:07.535353 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:07.526655    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/82ef0ce2-e735-491c-8489-509d7aeb82d4-maas-api-openshift-ca" pod "82ef0ce2-e735-491c-8489-509d7aeb82d4" (UID: "82ef0ce2-e735-491c-8489-509d7aeb82d4"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 13:15:07.543737 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:07.543696    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/82ef0ce2-e735-491c-8489-509d7aeb82d4-kube-api-access-cx82j" pod "82ef0ce2-e735-491c-8489-509d7aeb82d4" (UID: "82ef0ce2-e735-491c-8489-509d7aeb82d4"). InnerVolumeSpecName "kube-api-access-cx82j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:15:07.654331 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:07.609517    4943 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/82ef0ce2-e735-491c-8489-509d7aeb82d4-maas-api-xks-ca\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:15:07.667274 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:07.665554    4943 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/82ef0ce2-e735-491c-8489-509d7aeb82d4-maas-api-openshift-ca\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:15:07.667274 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:07.665582    4943 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cx82j\" (UniqueName: \"kubernetes.io/projected/82ef0ce2-e735-491c-8489-509d7aeb82d4-kube-api-access-cx82j\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:15:08.215079 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:08.215048    4943 scope.go:122] "RemoveContainer" containerID="9a09dfd59262a24e1d3e717d3522862f7e968b3bdb4ae05a749f9d0ca425cd31"
Oct 30 13:15:08.216240 ip-10-0-142-84 kubenswrapper[4943]: E1030 13:15:08.215694    4943 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w3-d0a18c-29889435-ds45l_odh-ai-gateway-infra(6719a2ae-c55d-477d-8853-f032599f9d45)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-d0a18c-29889435-ds45l" podUID="6719a2ae-c55d-477d-8853-f032599f9d45"
Oct 30 13:15:08.219310 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:08.218969    4943 scope.go:122] "RemoveContainer" containerID="b1a770a6e44dcf26ca00852f9b771c93a17373265128e074e8ab4c8147cbd6e4"
Oct 30 13:15:08.219310 ip-10-0-142-84 kubenswrapper[4943]: E1030 13:15:08.219231    4943 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w1-05942d-29889435-gvqj5_odh-ai-gateway-infra(fdaf8439-65b5-4e9a-b519-83d40aaf7307)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-05942d-29889435-gvqj5" podUID="fdaf8439-65b5-4e9a-b519-83d40aaf7307"
Oct 30 13:15:08.223028 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:08.220997    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-2e9a65a0-42gwd" event={"ID":"82ef0ce2-e735-491c-8489-509d7aeb82d4","Type":"ContainerDied","Data":"11ff7d0fee3e1aeebf5c068b00c9b8bf4486a3170488688d8e04c16a7960d347"}
Oct 30 13:15:08.223028 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:08.221045    4943 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="11ff7d0fee3e1aeebf5c068b00c9b8bf4486a3170488688d8e04c16a7960d347"
Oct 30 13:15:08.223028 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:08.221106    4943 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-2e9a65a0-42gwd"
Oct 30 13:15:08.233840 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:08.232683    4943 scope.go:122] "RemoveContainer" containerID="cd6cdcc885b81200fdb41dc892ee2500cac4d443520f804cb2ebbf9dbc204320"
Oct 30 13:15:08.233840 ip-10-0-142-84 kubenswrapper[4943]: E1030 13:15:08.232954    4943 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-29889435-75l6z_odh-ai-gateway-infra(bd5de1ce-17cd-4e87-bbac-766e14841a08)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889435-75l6z" podUID="bd5de1ce-17cd-4e87-bbac-766e14841a08"
Oct 30 13:15:08.236905 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:08.236617    4943 scope.go:122] "RemoveContainer" containerID="25223c0d82cc58dbf03dba699441936c3be600a550c013f29e54564d25eccddf"
Oct 30 13:15:08.236905 ip-10-0-142-84 kubenswrapper[4943]: E1030 13:15:08.236847    4943 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w6-ab3176-29889435-hnmkz_odh-ai-gateway-infra(8ec3dc68-1993-4289-bf82-f4e629c4d7ce)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-ab3176-29889435-hnmkz" podUID="8ec3dc68-1993-4289-bf82-f4e629c4d7ce"
Oct 30 13:15:08.504013 ip-10-0-142-84 kubenswrapper[4943]: E1030 13:15:08.503903    4943 cadvisor_stats_provider.go:569] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod82ef0ce2_e735_491c_8489_509d7aeb82d4.slice/crio-11ff7d0fee3e1aeebf5c068b00c9b8bf4486a3170488688d8e04c16a7960d347\": RecentStats: unable to find data in memory cache]"
Oct 30 13:15:14.258726 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:14.258690    4943 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-da16efe5-plpbz"]
Oct 30 13:15:14.259282 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:14.259253    4943 cpu_manager.go:414] "containerMap: removing container" podUID="82ef0ce2-e735-491c-8489-509d7aeb82d4" containerName="revoke-keys"
Oct 30 13:15:14.259282 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:14.259283    4943 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="82ef0ce2-e735-491c-8489-509d7aeb82d4" containerName="revoke-keys"
Oct 30 13:15:14.259407 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:14.259398    4943 memory_manager.go:365] "RemoveStaleState removing state" podUID="82ef0ce2-e735-491c-8489-509d7aeb82d4" containerName="revoke-keys"
Oct 30 13:15:14.264015 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:14.263995    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-da16efe5-plpbz"
Oct 30 13:15:14.285587 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:14.285562    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-da16efe5-plpbz"]
Oct 30 13:15:14.294753 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:14.294734    4943 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 13:15:14.345833 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:14.345803    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/5d850ac7-0bbd-466f-b3b2-c665eb07066f-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-da16efe5-plpbz\" (UID: \"5d850ac7-0bbd-466f-b3b2-c665eb07066f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-da16efe5-plpbz"
Oct 30 13:15:14.345957 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:14.345874    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hd74x\" (UniqueName: \"kubernetes.io/projected/5d850ac7-0bbd-466f-b3b2-c665eb07066f-kube-api-access-hd74x\") pod \"maas-api-revoke-sub-da16efe5-plpbz\" (UID: \"5d850ac7-0bbd-466f-b3b2-c665eb07066f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-da16efe5-plpbz"
Oct 30 13:15:14.346071 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:14.346045    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/5d850ac7-0bbd-466f-b3b2-c665eb07066f-maas-api-xks-ca\") pod \"maas-api-revoke-sub-da16efe5-plpbz\" (UID: \"5d850ac7-0bbd-466f-b3b2-c665eb07066f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-da16efe5-plpbz"
Oct 30 13:15:14.447527 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:14.447486    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/5d850ac7-0bbd-466f-b3b2-c665eb07066f-maas-api-xks-ca\") pod \"maas-api-revoke-sub-da16efe5-plpbz\" (UID: \"5d850ac7-0bbd-466f-b3b2-c665eb07066f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-da16efe5-plpbz"
Oct 30 13:15:14.447720 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:14.447559    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/5d850ac7-0bbd-466f-b3b2-c665eb07066f-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-da16efe5-plpbz\" (UID: \"5d850ac7-0bbd-466f-b3b2-c665eb07066f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-da16efe5-plpbz"
Oct 30 13:15:14.447720 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:14.447675    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hd74x\" (UniqueName: \"kubernetes.io/projected/5d850ac7-0bbd-466f-b3b2-c665eb07066f-kube-api-access-hd74x\") pod \"maas-api-revoke-sub-da16efe5-plpbz\" (UID: \"5d850ac7-0bbd-466f-b3b2-c665eb07066f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-da16efe5-plpbz"
Oct 30 13:15:14.448531 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:14.448504    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/5d850ac7-0bbd-466f-b3b2-c665eb07066f-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-da16efe5-plpbz\" (UID: \"5d850ac7-0bbd-466f-b3b2-c665eb07066f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-da16efe5-plpbz"
Oct 30 13:15:14.450093 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:14.450069    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/5d850ac7-0bbd-466f-b3b2-c665eb07066f-maas-api-xks-ca\") pod \"maas-api-revoke-sub-da16efe5-plpbz\" (UID: \"5d850ac7-0bbd-466f-b3b2-c665eb07066f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-da16efe5-plpbz"
Oct 30 13:15:14.473768 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:14.473737    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hd74x\" (UniqueName: \"kubernetes.io/projected/5d850ac7-0bbd-466f-b3b2-c665eb07066f-kube-api-access-hd74x\") pod \"maas-api-revoke-sub-da16efe5-plpbz\" (UID: \"5d850ac7-0bbd-466f-b3b2-c665eb07066f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-da16efe5-plpbz"
Oct 30 13:15:14.577974 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:14.577894    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-da16efe5-plpbz"
Oct 30 13:15:14.888071 ip-10-0-142-84 kubenswrapper[4943]: W1030 13:15:14.888040    4943 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5d850ac7_0bbd_466f_b3b2_c665eb07066f.slice/crio-f6cf55aca7432e771508253be73a49e7f717307c4dfa94a58f38ee50ca898aeb WatchSource:0}: Error finding container f6cf55aca7432e771508253be73a49e7f717307c4dfa94a58f38ee50ca898aeb: Status 404 returned error can't find the container with id f6cf55aca7432e771508253be73a49e7f717307c4dfa94a58f38ee50ca898aeb
Oct 30 13:15:14.888707 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:14.888676    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-da16efe5-plpbz"]
Oct 30 13:15:15.288086 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:15.288048    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-da16efe5-plpbz" event={"ID":"5d850ac7-0bbd-466f-b3b2-c665eb07066f","Type":"ContainerStarted","Data":"223db64525dfc66384575504caac98120a59bdb77baa254d263374796ab94914"}
Oct 30 13:15:15.288481 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:15.288111    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-da16efe5-plpbz" event={"ID":"5d850ac7-0bbd-466f-b3b2-c665eb07066f","Type":"ContainerStarted","Data":"f6cf55aca7432e771508253be73a49e7f717307c4dfa94a58f38ee50ca898aeb"}
Oct 30 13:15:15.335845 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:15.335789    4943 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-da16efe5-plpbz" podStartSLOduration=1.335775698 podStartE2EDuration="1.335775698s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:15:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:15:15.333453481 +0000 UTC m=+1091.121457223" watchObservedRunningTime="2026-10-30 13:15:15.335775698 +0000 UTC m=+1091.123779427"
Oct 30 13:15:16.296675 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:16.296641    4943 generic.go:356] "Generic (PLEG): container finished" podID="5d850ac7-0bbd-466f-b3b2-c665eb07066f" containerID="223db64525dfc66384575504caac98120a59bdb77baa254d263374796ab94914" exitCode=0
Oct 30 13:15:16.297095 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:16.296696    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-da16efe5-plpbz" event={"ID":"5d850ac7-0bbd-466f-b3b2-c665eb07066f","Type":"ContainerDied","Data":"223db64525dfc66384575504caac98120a59bdb77baa254d263374796ab94914"}
Oct 30 13:15:17.461683 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:17.461653    4943 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-da16efe5-plpbz"
Oct 30 13:15:17.596289 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:17.596195    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5d850ac7-0bbd-466f-b3b2-c665eb07066f-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/5d850ac7-0bbd-466f-b3b2-c665eb07066f-maas-api-openshift-ca\") pod \"5d850ac7-0bbd-466f-b3b2-c665eb07066f\" (UID: \"5d850ac7-0bbd-466f-b3b2-c665eb07066f\") "
Oct 30 13:15:17.596549 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:17.596340    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5d850ac7-0bbd-466f-b3b2-c665eb07066f-kube-api-access-hd74x\" (UniqueName: \"kubernetes.io/projected/5d850ac7-0bbd-466f-b3b2-c665eb07066f-kube-api-access-hd74x\") pod \"5d850ac7-0bbd-466f-b3b2-c665eb07066f\" (UID: \"5d850ac7-0bbd-466f-b3b2-c665eb07066f\") "
Oct 30 13:15:17.596549 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:17.596420    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/5d850ac7-0bbd-466f-b3b2-c665eb07066f-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/5d850ac7-0bbd-466f-b3b2-c665eb07066f-maas-api-xks-ca\") pod \"5d850ac7-0bbd-466f-b3b2-c665eb07066f\" (UID: \"5d850ac7-0bbd-466f-b3b2-c665eb07066f\") "
Oct 30 13:15:17.596713 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:17.596600    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5d850ac7-0bbd-466f-b3b2-c665eb07066f-maas-api-openshift-ca" pod "5d850ac7-0bbd-466f-b3b2-c665eb07066f" (UID: "5d850ac7-0bbd-466f-b3b2-c665eb07066f"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 13:15:17.596911 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:17.596882    4943 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/5d850ac7-0bbd-466f-b3b2-c665eb07066f-maas-api-openshift-ca\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:15:17.598512 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:17.598490    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5d850ac7-0bbd-466f-b3b2-c665eb07066f-maas-api-xks-ca" pod "5d850ac7-0bbd-466f-b3b2-c665eb07066f" (UID: "5d850ac7-0bbd-466f-b3b2-c665eb07066f"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 13:15:17.598823 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:17.598805    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5d850ac7-0bbd-466f-b3b2-c665eb07066f-kube-api-access-hd74x" pod "5d850ac7-0bbd-466f-b3b2-c665eb07066f" (UID: "5d850ac7-0bbd-466f-b3b2-c665eb07066f"). InnerVolumeSpecName "kube-api-access-hd74x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:15:17.698068 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:17.698033    4943 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/5d850ac7-0bbd-466f-b3b2-c665eb07066f-maas-api-xks-ca\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:15:17.698068 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:17.698066    4943 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hd74x\" (UniqueName: \"kubernetes.io/projected/5d850ac7-0bbd-466f-b3b2-c665eb07066f-kube-api-access-hd74x\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:15:18.311040 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:18.311010    4943 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-da16efe5-plpbz"
Oct 30 13:15:18.311199 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:18.311042    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-da16efe5-plpbz" event={"ID":"5d850ac7-0bbd-466f-b3b2-c665eb07066f","Type":"ContainerDied","Data":"f6cf55aca7432e771508253be73a49e7f717307c4dfa94a58f38ee50ca898aeb"}
Oct 30 13:15:18.311199 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:18.311093    4943 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f6cf55aca7432e771508253be73a49e7f717307c4dfa94a58f38ee50ca898aeb"
Oct 30 13:15:19.344012 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:19.343978    4943 scope.go:122] "RemoveContainer" containerID="25223c0d82cc58dbf03dba699441936c3be600a550c013f29e54564d25eccddf"
Oct 30 13:15:20.330216 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:20.330179    4943 generic.go:356] "Generic (PLEG): container finished" podID="8ec3dc68-1993-4289-bf82-f4e629c4d7ce" containerID="aa40bed31fb4fac007ebd149f6dbcb8f6b85f477754b00ed29dcb20309198023" exitCode=60
Oct 30 13:15:20.330416 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:20.330274    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-ab3176-29889435-hnmkz" event={"ID":"8ec3dc68-1993-4289-bf82-f4e629c4d7ce","Type":"ContainerDied","Data":"aa40bed31fb4fac007ebd149f6dbcb8f6b85f477754b00ed29dcb20309198023"}
Oct 30 13:15:20.330416 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:20.330353    4943 scope.go:122] "RemoveContainer" containerID="25223c0d82cc58dbf03dba699441936c3be600a550c013f29e54564d25eccddf"
Oct 30 13:15:20.330719 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:20.330701    4943 scope.go:122] "RemoveContainer" containerID="aa40bed31fb4fac007ebd149f6dbcb8f6b85f477754b00ed29dcb20309198023"
Oct 30 13:15:20.330993 ip-10-0-142-84 kubenswrapper[4943]: E1030 13:15:20.330969    4943 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w6-ab3176-29889435-hnmkz_odh-ai-gateway-infra(8ec3dc68-1993-4289-bf82-f4e629c4d7ce)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-ab3176-29889435-hnmkz" podUID="8ec3dc68-1993-4289-bf82-f4e629c4d7ce"
Oct 30 13:15:20.343997 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:20.343591    4943 scope.go:122] "RemoveContainer" containerID="b1a770a6e44dcf26ca00852f9b771c93a17373265128e074e8ab4c8147cbd6e4"
Oct 30 13:15:20.343997 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:20.343723    4943 scope.go:122] "RemoveContainer" containerID="9a09dfd59262a24e1d3e717d3522862f7e968b3bdb4ae05a749f9d0ca425cd31"
Oct 30 13:15:21.343139 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:21.343112    4943 scope.go:122] "RemoveContainer" containerID="cd6cdcc885b81200fdb41dc892ee2500cac4d443520f804cb2ebbf9dbc204320"
Oct 30 13:15:21.344107 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:21.344077    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-d0a18c-29889435-ds45l" event={"ID":"6719a2ae-c55d-477d-8853-f032599f9d45","Type":"ContainerStarted","Data":"86e59a956548bfd720b2fd0dfb8e904e31a61ae47d1465c77aa0406046c00f11"}
Oct 30 13:15:21.346627 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:21.346600    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-05942d-29889435-gvqj5" event={"ID":"fdaf8439-65b5-4e9a-b519-83d40aaf7307","Type":"ContainerStarted","Data":"64499fcf0c852f9c7a41c39c574a3bd85b178b5ae4748d11d0cc9c43660c6a6c"}
Oct 30 13:15:21.384069 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:21.384037    4943 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-f55w2"]
Oct 30 13:15:21.385187 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:21.385170    4943 cpu_manager.go:414] "containerMap: removing container" podUID="5d850ac7-0bbd-466f-b3b2-c665eb07066f" containerName="revoke-keys"
Oct 30 13:15:21.385292 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:21.385191    4943 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5d850ac7-0bbd-466f-b3b2-c665eb07066f" containerName="revoke-keys"
Oct 30 13:15:21.385385 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:21.385357    4943 memory_manager.go:365] "RemoveStaleState removing state" podUID="5d850ac7-0bbd-466f-b3b2-c665eb07066f" containerName="revoke-keys"
Oct 30 13:15:21.398034 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:21.394493    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-f55w2"
Oct 30 13:15:21.404292 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:21.403362    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-f55w2"]
Oct 30 13:15:21.407746 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:21.407691    4943 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-d0a18c-29889435-ds45l" podStartSLOduration=17.643184582 podStartE2EDuration="21.407674987s" totalImagesPullingTime="3.764490405s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:15:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:15:21.401360285 +0000 UTC m=+1097.189364042" watchObservedRunningTime="2026-10-30 13:15:21.407674987 +0000 UTC m=+1097.195678718"
Oct 30 13:15:21.412337 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:21.412315    4943 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-ab3176-29889435-hnmkz"]
Oct 30 13:15:21.419169 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:21.419096    4943 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w1-05942d-openshift-default-dockercfg-hrnkn\""
Oct 30 13:15:21.546695 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:21.545939    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/663524af-aa09-44fe-8186-654cb9668d27-credential-socket\") pod \"e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-f55w2\" (UID: \"663524af-aa09-44fe-8186-654cb9668d27\") " pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-f55w2"
Oct 30 13:15:21.546695 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:21.546041    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/663524af-aa09-44fe-8186-654cb9668d27-istio-token\") pod \"e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-f55w2\" (UID: \"663524af-aa09-44fe-8186-654cb9668d27\") " pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-f55w2"
Oct 30 13:15:21.546695 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:21.546102    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/663524af-aa09-44fe-8186-654cb9668d27-istio-podinfo\") pod \"e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-f55w2\" (UID: \"663524af-aa09-44fe-8186-654cb9668d27\") " pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-f55w2"
Oct 30 13:15:21.546695 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:21.546157    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/663524af-aa09-44fe-8186-654cb9668d27-istio-envoy\") pod \"e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-f55w2\" (UID: \"663524af-aa09-44fe-8186-654cb9668d27\") " pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-f55w2"
Oct 30 13:15:21.546695 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:21.546185    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/663524af-aa09-44fe-8186-654cb9668d27-istiod-ca-cert\") pod \"e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-f55w2\" (UID: \"663524af-aa09-44fe-8186-654cb9668d27\") " pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-f55w2"
Oct 30 13:15:21.546695 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:21.546279    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/663524af-aa09-44fe-8186-654cb9668d27-workload-socket\") pod \"e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-f55w2\" (UID: \"663524af-aa09-44fe-8186-654cb9668d27\") " pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-f55w2"
Oct 30 13:15:21.546695 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:21.546318    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/663524af-aa09-44fe-8186-654cb9668d27-workload-certs\") pod \"e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-f55w2\" (UID: \"663524af-aa09-44fe-8186-654cb9668d27\") " pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-f55w2"
Oct 30 13:15:21.546695 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:21.546367    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jzvdf\" (UniqueName: \"kubernetes.io/projected/663524af-aa09-44fe-8186-654cb9668d27-kube-api-access-jzvdf\") pod \"e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-f55w2\" (UID: \"663524af-aa09-44fe-8186-654cb9668d27\") " pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-f55w2"
Oct 30 13:15:21.546695 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:21.546405    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/663524af-aa09-44fe-8186-654cb9668d27-istio-data\") pod \"e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-f55w2\" (UID: \"663524af-aa09-44fe-8186-654cb9668d27\") " pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-f55w2"
Oct 30 13:15:21.651332 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:21.648926    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/663524af-aa09-44fe-8186-654cb9668d27-istiod-ca-cert\") pod \"e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-f55w2\" (UID: \"663524af-aa09-44fe-8186-654cb9668d27\") " pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-f55w2"
Oct 30 13:15:21.651332 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:21.647213    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/663524af-aa09-44fe-8186-654cb9668d27-istiod-ca-cert\") pod \"e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-f55w2\" (UID: \"663524af-aa09-44fe-8186-654cb9668d27\") " pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-f55w2"
Oct 30 13:15:21.651332 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:21.649124    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/663524af-aa09-44fe-8186-654cb9668d27-workload-socket\") pod \"e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-f55w2\" (UID: \"663524af-aa09-44fe-8186-654cb9668d27\") " pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-f55w2"
Oct 30 13:15:21.651332 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:21.649480    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/663524af-aa09-44fe-8186-654cb9668d27-workload-socket\") pod \"e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-f55w2\" (UID: \"663524af-aa09-44fe-8186-654cb9668d27\") " pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-f55w2"
Oct 30 13:15:21.651332 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:21.649901    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/663524af-aa09-44fe-8186-654cb9668d27-workload-certs\") pod \"e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-f55w2\" (UID: \"663524af-aa09-44fe-8186-654cb9668d27\") " pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-f55w2"
Oct 30 13:15:21.651332 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:21.649584    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/663524af-aa09-44fe-8186-654cb9668d27-workload-certs\") pod \"e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-f55w2\" (UID: \"663524af-aa09-44fe-8186-654cb9668d27\") " pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-f55w2"
Oct 30 13:15:21.651332 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:21.650007    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jzvdf\" (UniqueName: \"kubernetes.io/projected/663524af-aa09-44fe-8186-654cb9668d27-kube-api-access-jzvdf\") pod \"e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-f55w2\" (UID: \"663524af-aa09-44fe-8186-654cb9668d27\") " pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-f55w2"
Oct 30 13:15:21.651332 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:21.650051    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/663524af-aa09-44fe-8186-654cb9668d27-istio-data\") pod \"e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-f55w2\" (UID: \"663524af-aa09-44fe-8186-654cb9668d27\") " pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-f55w2"
Oct 30 13:15:21.651332 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:21.650423    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/663524af-aa09-44fe-8186-654cb9668d27-istio-data\") pod \"e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-f55w2\" (UID: \"663524af-aa09-44fe-8186-654cb9668d27\") " pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-f55w2"
Oct 30 13:15:21.651332 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:21.650121    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/663524af-aa09-44fe-8186-654cb9668d27-credential-socket\") pod \"e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-f55w2\" (UID: \"663524af-aa09-44fe-8186-654cb9668d27\") " pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-f55w2"
Oct 30 13:15:21.651332 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:21.650571    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/663524af-aa09-44fe-8186-654cb9668d27-istio-token\") pod \"e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-f55w2\" (UID: \"663524af-aa09-44fe-8186-654cb9668d27\") " pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-f55w2"
Oct 30 13:15:21.652030 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:21.650658    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/663524af-aa09-44fe-8186-654cb9668d27-istio-podinfo\") pod \"e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-f55w2\" (UID: \"663524af-aa09-44fe-8186-654cb9668d27\") " pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-f55w2"
Oct 30 13:15:21.652030 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:21.650756    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/663524af-aa09-44fe-8186-654cb9668d27-credential-socket\") pod \"e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-f55w2\" (UID: \"663524af-aa09-44fe-8186-654cb9668d27\") " pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-f55w2"
Oct 30 13:15:21.652030 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:21.651508    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/663524af-aa09-44fe-8186-654cb9668d27-istio-envoy\") pod \"e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-f55w2\" (UID: \"663524af-aa09-44fe-8186-654cb9668d27\") " pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-f55w2"
Oct 30 13:15:21.654319 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:21.654292    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/663524af-aa09-44fe-8186-654cb9668d27-istio-envoy\") pod \"e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-f55w2\" (UID: \"663524af-aa09-44fe-8186-654cb9668d27\") " pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-f55w2"
Oct 30 13:15:21.655049 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:21.655026    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/663524af-aa09-44fe-8186-654cb9668d27-istio-podinfo\") pod \"e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-f55w2\" (UID: \"663524af-aa09-44fe-8186-654cb9668d27\") " pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-f55w2"
Oct 30 13:15:21.678771 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:21.677369    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/663524af-aa09-44fe-8186-654cb9668d27-istio-token\") pod \"e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-f55w2\" (UID: \"663524af-aa09-44fe-8186-654cb9668d27\") " pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-f55w2"
Oct 30 13:15:21.678771 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:21.677474    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jzvdf\" (UniqueName: \"kubernetes.io/projected/663524af-aa09-44fe-8186-654cb9668d27-kube-api-access-jzvdf\") pod \"e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-f55w2\" (UID: \"663524af-aa09-44fe-8186-654cb9668d27\") " pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-f55w2"
Oct 30 13:15:21.712185 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:21.705421    4943 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-ab3176-29889435-hnmkz"
Oct 30 13:15:21.757117 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:21.755332    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8ec3dc68-1993-4289-bf82-f4e629c4d7ce-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8ec3dc68-1993-4289-bf82-f4e629c4d7ce-maas-api-service-ca\") pod \"8ec3dc68-1993-4289-bf82-f4e629c4d7ce\" (UID: \"8ec3dc68-1993-4289-bf82-f4e629c4d7ce\") "
Oct 30 13:15:21.757117 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:21.755441    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8ec3dc68-1993-4289-bf82-f4e629c4d7ce-kube-api-access-kgt9d\" (UniqueName: \"kubernetes.io/projected/8ec3dc68-1993-4289-bf82-f4e629c4d7ce-kube-api-access-kgt9d\") pod \"8ec3dc68-1993-4289-bf82-f4e629c4d7ce\" (UID: \"8ec3dc68-1993-4289-bf82-f4e629c4d7ce\") "
Oct 30 13:15:21.757833 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:21.757778    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8ec3dc68-1993-4289-bf82-f4e629c4d7ce-maas-api-service-ca" pod "8ec3dc68-1993-4289-bf82-f4e629c4d7ce" (UID: "8ec3dc68-1993-4289-bf82-f4e629c4d7ce"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 13:15:21.783510 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:21.783468    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8ec3dc68-1993-4289-bf82-f4e629c4d7ce-kube-api-access-kgt9d" pod "8ec3dc68-1993-4289-bf82-f4e629c4d7ce" (UID: "8ec3dc68-1993-4289-bf82-f4e629c4d7ce"). InnerVolumeSpecName "kube-api-access-kgt9d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:15:21.808902 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:21.808584    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-f55w2"
Oct 30 13:15:21.900613 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:21.859195    4943 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8ec3dc68-1993-4289-bf82-f4e629c4d7ce-maas-api-service-ca\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:15:21.900613 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:21.859237    4943 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kgt9d\" (UniqueName: \"kubernetes.io/projected/8ec3dc68-1993-4289-bf82-f4e629c4d7ce-kube-api-access-kgt9d\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:15:22.376293 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:22.374554    4943 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-ab3176-29889435-hnmkz"
Oct 30 13:15:22.382320 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:22.379343    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-f55w2"]
Oct 30 13:15:22.382320 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:22.379423    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889435-75l6z" event={"ID":"bd5de1ce-17cd-4e87-bbac-766e14841a08","Type":"ContainerStarted","Data":"1863c1a7d5522703c6e9f7fc524a4297993a8c9df9c03f114696daab90efb532"}
Oct 30 13:15:22.382320 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:22.379461    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-ab3176-29889435-hnmkz" event={"ID":"8ec3dc68-1993-4289-bf82-f4e629c4d7ce","Type":"ContainerDied","Data":"424695806e534b4dbccf35f56629d72a7c9cd84fbe1ae31cd7e9f4a83f20c78e"}
Oct 30 13:15:22.382320 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:22.379499    4943 scope.go:122] "RemoveContainer" containerID="aa40bed31fb4fac007ebd149f6dbcb8f6b85f477754b00ed29dcb20309198023"
Oct 30 13:15:22.545005 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:22.544960    4943 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 13:15:22.545251 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:22.545226    4943 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 13:15:22.545421 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:22.545400    4943 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 13:15:22.551427 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:22.551409    4943 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-d0a18c-29889435-ds45l"]
Oct 30 13:15:22.561152 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:22.561123    4943 generic.go:356] "Generic (PLEG): container finished" podID="6719a2ae-c55d-477d-8853-f032599f9d45" containerID="86e59a956548bfd720b2fd0dfb8e904e31a61ae47d1465c77aa0406046c00f11" exitCode=60
Oct 30 13:15:22.561426 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:22.561405    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-d0a18c-29889435-ds45l" event={"ID":"6719a2ae-c55d-477d-8853-f032599f9d45","Type":"ContainerDied","Data":"86e59a956548bfd720b2fd0dfb8e904e31a61ae47d1465c77aa0406046c00f11"}
Oct 30 13:15:22.571178 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:22.571158    4943 scope.go:122] "RemoveContainer" containerID="9a09dfd59262a24e1d3e717d3522862f7e968b3bdb4ae05a749f9d0ca425cd31"
Oct 30 13:15:22.575177 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:22.575117    4943 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-05942d-29889435-gvqj5"]
Oct 30 13:15:22.577480 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:22.577407    4943 generic.go:356] "Generic (PLEG): container finished" podID="fdaf8439-65b5-4e9a-b519-83d40aaf7307" containerID="64499fcf0c852f9c7a41c39c574a3bd85b178b5ae4748d11d0cc9c43660c6a6c" exitCode=60
Oct 30 13:15:22.577814 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:22.577455    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-05942d-29889435-gvqj5" event={"ID":"fdaf8439-65b5-4e9a-b519-83d40aaf7307","Type":"ContainerDied","Data":"64499fcf0c852f9c7a41c39c574a3bd85b178b5ae4748d11d0cc9c43660c6a6c"}
Oct 30 13:15:22.584584 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:22.584359    4943 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889435-75l6z" podStartSLOduration=18.908744687 podStartE2EDuration="22.584343226s" totalImagesPullingTime="3.675598539s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:15:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:15:22.558627977 +0000 UTC m=+1098.346631708" watchObservedRunningTime="2026-10-30 13:15:22.584343226 +0000 UTC m=+1098.372346957"
Oct 30 13:15:22.702373 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:22.700608    4943 scope.go:122] "RemoveContainer" containerID="b1a770a6e44dcf26ca00852f9b771c93a17373265128e074e8ab4c8147cbd6e4"
Oct 30 13:15:22.743503 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:22.743440    4943 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-ab3176-29889435-hnmkz"]
Oct 30 13:15:22.750229 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:22.749649    4943 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-ab3176-29889435-hnmkz"]
Oct 30 13:15:22.944288 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:22.937604    4943 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-05942d-29889435-gvqj5"
Oct 30 13:15:23.007306 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:23.006960    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fdaf8439-65b5-4e9a-b519-83d40aaf7307-kube-api-access-dk48v\" (UniqueName: \"kubernetes.io/projected/fdaf8439-65b5-4e9a-b519-83d40aaf7307-kube-api-access-dk48v\") pod \"fdaf8439-65b5-4e9a-b519-83d40aaf7307\" (UID: \"fdaf8439-65b5-4e9a-b519-83d40aaf7307\") "
Oct 30 13:15:23.007306 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:23.007180    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/fdaf8439-65b5-4e9a-b519-83d40aaf7307-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fdaf8439-65b5-4e9a-b519-83d40aaf7307-maas-api-service-ca\") pod \"fdaf8439-65b5-4e9a-b519-83d40aaf7307\" (UID: \"fdaf8439-65b5-4e9a-b519-83d40aaf7307\") "
Oct 30 13:15:23.007613 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:23.007593    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fdaf8439-65b5-4e9a-b519-83d40aaf7307-maas-api-service-ca" pod "fdaf8439-65b5-4e9a-b519-83d40aaf7307" (UID: "fdaf8439-65b5-4e9a-b519-83d40aaf7307"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 13:15:23.010543 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:23.009669    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fdaf8439-65b5-4e9a-b519-83d40aaf7307-kube-api-access-dk48v" pod "fdaf8439-65b5-4e9a-b519-83d40aaf7307" (UID: "fdaf8439-65b5-4e9a-b519-83d40aaf7307"). InnerVolumeSpecName "kube-api-access-dk48v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:15:23.019295 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:23.019254    4943 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-d0a18c-29889435-ds45l"
Oct 30 13:15:23.110351 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:23.108204    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6719a2ae-c55d-477d-8853-f032599f9d45-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6719a2ae-c55d-477d-8853-f032599f9d45-maas-api-service-ca\") pod \"6719a2ae-c55d-477d-8853-f032599f9d45\" (UID: \"6719a2ae-c55d-477d-8853-f032599f9d45\") "
Oct 30 13:15:23.110351 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:23.108592    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6719a2ae-c55d-477d-8853-f032599f9d45-maas-api-service-ca" pod "6719a2ae-c55d-477d-8853-f032599f9d45" (UID: "6719a2ae-c55d-477d-8853-f032599f9d45"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 13:15:23.110351 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:23.108741    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6719a2ae-c55d-477d-8853-f032599f9d45-kube-api-access-lhfph\" (UniqueName: \"kubernetes.io/projected/6719a2ae-c55d-477d-8853-f032599f9d45-kube-api-access-lhfph\") pod \"6719a2ae-c55d-477d-8853-f032599f9d45\" (UID: \"6719a2ae-c55d-477d-8853-f032599f9d45\") "
Oct 30 13:15:23.110351 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:23.109183    4943 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fdaf8439-65b5-4e9a-b519-83d40aaf7307-maas-api-service-ca\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:15:23.110351 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:23.109215    4943 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6719a2ae-c55d-477d-8853-f032599f9d45-maas-api-service-ca\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:15:23.110351 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:23.109235    4943 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dk48v\" (UniqueName: \"kubernetes.io/projected/fdaf8439-65b5-4e9a-b519-83d40aaf7307-kube-api-access-dk48v\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:15:23.111668 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:23.111635    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6719a2ae-c55d-477d-8853-f032599f9d45-kube-api-access-lhfph" pod "6719a2ae-c55d-477d-8853-f032599f9d45" (UID: "6719a2ae-c55d-477d-8853-f032599f9d45"). InnerVolumeSpecName "kube-api-access-lhfph". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:15:23.210625 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:23.210588    4943 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lhfph\" (UniqueName: \"kubernetes.io/projected/6719a2ae-c55d-477d-8853-f032599f9d45-kube-api-access-lhfph\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:15:23.557718 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:23.557679    4943 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29889435-75l6z"]
Oct 30 13:15:23.591443 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:23.589246    4943 generic.go:356] "Generic (PLEG): container finished" podID="bd5de1ce-17cd-4e87-bbac-766e14841a08" containerID="1863c1a7d5522703c6e9f7fc524a4297993a8c9df9c03f114696daab90efb532" exitCode=60
Oct 30 13:15:23.591443 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:23.589384    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889435-75l6z" event={"ID":"bd5de1ce-17cd-4e87-bbac-766e14841a08","Type":"ContainerDied","Data":"1863c1a7d5522703c6e9f7fc524a4297993a8c9df9c03f114696daab90efb532"}
Oct 30 13:15:23.591443 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:23.589444    4943 scope.go:122] "RemoveContainer" containerID="cd6cdcc885b81200fdb41dc892ee2500cac4d443520f804cb2ebbf9dbc204320"
Oct 30 13:15:23.593820 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:23.593797    4943 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-d0a18c-29889435-ds45l"
Oct 30 13:15:23.593967 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:23.593837    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-d0a18c-29889435-ds45l" event={"ID":"6719a2ae-c55d-477d-8853-f032599f9d45","Type":"ContainerDied","Data":"47623692b4a164e330c6f7357f7b18c9496ad8bf06dc059ff37642b9d5cd690b"}
Oct 30 13:15:23.595677 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:23.595658    4943 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-05942d-29889435-gvqj5"
Oct 30 13:15:23.596464 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:23.595704    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-05942d-29889435-gvqj5" event={"ID":"fdaf8439-65b5-4e9a-b519-83d40aaf7307","Type":"ContainerDied","Data":"9c6b7303218d632c3c863b190193671a672f110ce029e68a76b68ac2852b80ca"}
Oct 30 13:15:23.597734 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:23.597699    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-f55w2" event={"ID":"663524af-aa09-44fe-8186-654cb9668d27","Type":"ContainerStarted","Data":"a7aab891628438e6299302e8f696c682fa173c6902aa23e763ea7aa6424ad66a"}
Oct 30 13:15:23.597834 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:23.597753    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-f55w2" event={"ID":"663524af-aa09-44fe-8186-654cb9668d27","Type":"ContainerStarted","Data":"35eff802ed42ee69a610841e96391731fddf9bd67b51db2310118f3539f87166"}
Oct 30 13:15:23.634175 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:23.634151    4943 scope.go:122] "RemoveContainer" containerID="86e59a956548bfd720b2fd0dfb8e904e31a61ae47d1465c77aa0406046c00f11"
Oct 30 13:15:23.652083 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:23.652057    4943 scope.go:122] "RemoveContainer" containerID="64499fcf0c852f9c7a41c39c574a3bd85b178b5ae4748d11d0cc9c43660c6a6c"
Oct 30 13:15:23.731556 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:23.713384    4943 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-05942d-29889435-gvqj5"]
Oct 30 13:15:23.731556 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:23.730408    4943 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-05942d-29889435-gvqj5"]
Oct 30 13:15:23.774909 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:23.774695    4943 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-f55w2" podStartSLOduration=2.774673211 podStartE2EDuration="2.774673211s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:15:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:15:23.772295346 +0000 UTC m=+1099.560299076" watchObservedRunningTime="2026-10-30 13:15:23.774673211 +0000 UTC m=+1099.562676944"
Oct 30 13:15:23.812559 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:23.812484    4943 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-f55w2"
Oct 30 13:15:23.814024 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:23.813998    4943 patch_prober.go:29] interesting pod/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-f55w2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.81:15021/healthz/ready\": dial tcp 10.132.0.81:15021: connect: connection refused" start-of-body=
Oct 30 13:15:23.814158 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:23.814049    4943 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-f55w2" podUID="663524af-aa09-44fe-8186-654cb9668d27" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.81:15021/healthz/ready\": dial tcp 10.132.0.81:15021: connect: connection refused"
Oct 30 13:15:23.822594 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:23.822568    4943 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-d0a18c-29889435-ds45l"]
Oct 30 13:15:23.826806 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:23.826784    4943 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-d0a18c-29889435-ds45l"]
Oct 30 13:15:23.901967 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:23.901939    4943 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889435-75l6z"
Oct 30 13:15:23.928451 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:23.928423    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/bd5de1ce-17cd-4e87-bbac-766e14841a08-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bd5de1ce-17cd-4e87-bbac-766e14841a08-maas-api-service-ca\") pod \"bd5de1ce-17cd-4e87-bbac-766e14841a08\" (UID: \"bd5de1ce-17cd-4e87-bbac-766e14841a08\") "
Oct 30 13:15:23.928619 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:23.928482    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/bd5de1ce-17cd-4e87-bbac-766e14841a08-kube-api-access-fvm5x\" (UniqueName: \"kubernetes.io/projected/bd5de1ce-17cd-4e87-bbac-766e14841a08-kube-api-access-fvm5x\") pod \"bd5de1ce-17cd-4e87-bbac-766e14841a08\" (UID: \"bd5de1ce-17cd-4e87-bbac-766e14841a08\") "
Oct 30 13:15:23.928796 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:23.928771    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bd5de1ce-17cd-4e87-bbac-766e14841a08-maas-api-service-ca" pod "bd5de1ce-17cd-4e87-bbac-766e14841a08" (UID: "bd5de1ce-17cd-4e87-bbac-766e14841a08"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 13:15:23.930611 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:23.930583    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bd5de1ce-17cd-4e87-bbac-766e14841a08-kube-api-access-fvm5x" pod "bd5de1ce-17cd-4e87-bbac-766e14841a08" (UID: "bd5de1ce-17cd-4e87-bbac-766e14841a08"). InnerVolumeSpecName "kube-api-access-fvm5x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:15:24.029566 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:24.029524    4943 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bd5de1ce-17cd-4e87-bbac-766e14841a08-maas-api-service-ca\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:15:24.029713 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:24.029558    4943 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fvm5x\" (UniqueName: \"kubernetes.io/projected/bd5de1ce-17cd-4e87-bbac-766e14841a08-kube-api-access-fvm5x\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:15:24.350589 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:24.350554    4943 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6719a2ae-c55d-477d-8853-f032599f9d45" path="/var/lib/kubelet/pods/6719a2ae-c55d-477d-8853-f032599f9d45/volumes"
Oct 30 13:15:24.350963 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:24.350949    4943 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8ec3dc68-1993-4289-bf82-f4e629c4d7ce" path="/var/lib/kubelet/pods/8ec3dc68-1993-4289-bf82-f4e629c4d7ce/volumes"
Oct 30 13:15:24.351333 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:24.351321    4943 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="fdaf8439-65b5-4e9a-b519-83d40aaf7307" path="/var/lib/kubelet/pods/fdaf8439-65b5-4e9a-b519-83d40aaf7307/volumes"
Oct 30 13:15:24.606818 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:24.606734    4943 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889435-75l6z"
Oct 30 13:15:24.606818 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:24.606779    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889435-75l6z" event={"ID":"bd5de1ce-17cd-4e87-bbac-766e14841a08","Type":"ContainerDied","Data":"39b834608cea5321f9120793febb86e7d7cbe31871a70ad8bba0b1e367ef3917"}
Oct 30 13:15:24.607248 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:24.606838    4943 scope.go:122] "RemoveContainer" containerID="1863c1a7d5522703c6e9f7fc524a4297993a8c9df9c03f114696daab90efb532"
Oct 30 13:15:24.687288 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:24.681624    4943 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29889435-75l6z"]
Oct 30 13:15:24.690221 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:24.690165    4943 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29889435-75l6z"]
Oct 30 13:15:24.809961 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:24.809922    4943 patch_prober.go:29] interesting pod/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-f55w2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.81:15021/healthz/ready\": dial tcp 10.132.0.81:15021: connect: connection refused" start-of-body=
Oct 30 13:15:24.810130 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:24.809993    4943 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-f55w2" podUID="663524af-aa09-44fe-8186-654cb9668d27" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.81:15021/healthz/ready\": dial tcp 10.132.0.81:15021: connect: connection refused"
Oct 30 13:15:25.809483 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:25.809448    4943 patch_prober.go:29] interesting pod/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-f55w2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.81:15021/healthz/ready\": dial tcp 10.132.0.81:15021: connect: connection refused" start-of-body=
Oct 30 13:15:25.809905 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:25.809518    4943 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-f55w2" podUID="663524af-aa09-44fe-8186-654cb9668d27" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.81:15021/healthz/ready\": dial tcp 10.132.0.81:15021: connect: connection refused"
Oct 30 13:15:26.353558 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:26.353469    4943 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="bd5de1ce-17cd-4e87-bbac-766e14841a08" path="/var/lib/kubelet/pods/bd5de1ce-17cd-4e87-bbac-766e14841a08/volumes"
Oct 30 13:15:26.814308 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:26.814095    4943 patch_prober.go:29] interesting pod/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-f55w2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.81:15021/healthz/ready\": dial tcp 10.132.0.81:15021: connect: connection refused" start-of-body=
Oct 30 13:15:26.814308 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:26.814184    4943 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-f55w2" podUID="663524af-aa09-44fe-8186-654cb9668d27" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.81:15021/healthz/ready\": dial tcp 10.132.0.81:15021: connect: connection refused"
Oct 30 13:15:27.809537 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:27.809502    4943 patch_prober.go:29] interesting pod/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-f55w2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.81:15021/healthz/ready\": dial tcp 10.132.0.81:15021: connect: connection refused" start-of-body=
Oct 30 13:15:27.809718 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:27.809556    4943 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-f55w2" podUID="663524af-aa09-44fe-8186-654cb9668d27" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.81:15021/healthz/ready\": dial tcp 10.132.0.81:15021: connect: connection refused"
Oct 30 13:15:28.809650 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:28.809599    4943 patch_prober.go:29] interesting pod/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-f55w2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.81:15021/healthz/ready\": dial tcp 10.132.0.81:15021: connect: connection refused" start-of-body=
Oct 30 13:15:28.810059 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:28.809687    4943 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-f55w2" podUID="663524af-aa09-44fe-8186-654cb9668d27" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.81:15021/healthz/ready\": dial tcp 10.132.0.81:15021: connect: connection refused"
Oct 30 13:15:29.380842 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:29.380803    4943 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-843da301-npccp"]
Oct 30 13:15:29.381484 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:29.381424    4943 cpu_manager.go:414] "containerMap: removing container" podUID="fdaf8439-65b5-4e9a-b519-83d40aaf7307" containerName="cleanup"
Oct 30 13:15:29.381557 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:29.381484    4943 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fdaf8439-65b5-4e9a-b519-83d40aaf7307" containerName="cleanup"
Oct 30 13:15:29.381557 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:29.381504    4943 cpu_manager.go:414] "containerMap: removing container" podUID="bd5de1ce-17cd-4e87-bbac-766e14841a08" containerName="cleanup"
Oct 30 13:15:29.381557 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:29.381510    4943 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="bd5de1ce-17cd-4e87-bbac-766e14841a08" containerName="cleanup"
Oct 30 13:15:29.381557 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:29.381520    4943 cpu_manager.go:414] "containerMap: removing container" podUID="8ec3dc68-1993-4289-bf82-f4e629c4d7ce" containerName="cleanup"
Oct 30 13:15:29.381557 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:29.381526    4943 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8ec3dc68-1993-4289-bf82-f4e629c4d7ce" containerName="cleanup"
Oct 30 13:15:29.381557 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:29.381552    4943 cpu_manager.go:414] "containerMap: removing container" podUID="fdaf8439-65b5-4e9a-b519-83d40aaf7307" containerName="cleanup"
Oct 30 13:15:29.381729 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:29.381560    4943 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fdaf8439-65b5-4e9a-b519-83d40aaf7307" containerName="cleanup"
Oct 30 13:15:29.381729 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:29.381580    4943 cpu_manager.go:414] "containerMap: removing container" podUID="8ec3dc68-1993-4289-bf82-f4e629c4d7ce" containerName="cleanup"
Oct 30 13:15:29.381729 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:29.381587    4943 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8ec3dc68-1993-4289-bf82-f4e629c4d7ce" containerName="cleanup"
Oct 30 13:15:29.381729 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:29.381598    4943 cpu_manager.go:414] "containerMap: removing container" podUID="6719a2ae-c55d-477d-8853-f032599f9d45" containerName="cleanup"
Oct 30 13:15:29.381729 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:29.381605    4943 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6719a2ae-c55d-477d-8853-f032599f9d45" containerName="cleanup"
Oct 30 13:15:29.381729 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:29.381615    4943 cpu_manager.go:414] "containerMap: removing container" podUID="6719a2ae-c55d-477d-8853-f032599f9d45" containerName="cleanup"
Oct 30 13:15:29.381729 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:29.381621    4943 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6719a2ae-c55d-477d-8853-f032599f9d45" containerName="cleanup"
Oct 30 13:15:29.381729 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:29.381634    4943 cpu_manager.go:414] "containerMap: removing container" podUID="bd5de1ce-17cd-4e87-bbac-766e14841a08" containerName="cleanup"
Oct 30 13:15:29.381729 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:29.381641    4943 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="bd5de1ce-17cd-4e87-bbac-766e14841a08" containerName="cleanup"
Oct 30 13:15:29.381729 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:29.381674    4943 cpu_manager.go:414] "containerMap: removing container" podUID="bd5de1ce-17cd-4e87-bbac-766e14841a08" containerName="cleanup"
Oct 30 13:15:29.381729 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:29.381681    4943 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="bd5de1ce-17cd-4e87-bbac-766e14841a08" containerName="cleanup"
Oct 30 13:15:29.382015 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:29.381820    4943 memory_manager.go:365] "RemoveStaleState removing state" podUID="6719a2ae-c55d-477d-8853-f032599f9d45" containerName="cleanup"
Oct 30 13:15:29.382015 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:29.381830    4943 memory_manager.go:365] "RemoveStaleState removing state" podUID="fdaf8439-65b5-4e9a-b519-83d40aaf7307" containerName="cleanup"
Oct 30 13:15:29.382015 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:29.381836    4943 memory_manager.go:365] "RemoveStaleState removing state" podUID="fdaf8439-65b5-4e9a-b519-83d40aaf7307" containerName="cleanup"
Oct 30 13:15:29.382015 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:29.381844    4943 memory_manager.go:365] "RemoveStaleState removing state" podUID="bd5de1ce-17cd-4e87-bbac-766e14841a08" containerName="cleanup"
Oct 30 13:15:29.382015 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:29.381849    4943 memory_manager.go:365] "RemoveStaleState removing state" podUID="8ec3dc68-1993-4289-bf82-f4e629c4d7ce" containerName="cleanup"
Oct 30 13:15:29.382015 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:29.381857    4943 memory_manager.go:365] "RemoveStaleState removing state" podUID="8ec3dc68-1993-4289-bf82-f4e629c4d7ce" containerName="cleanup"
Oct 30 13:15:29.382015 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:29.381862    4943 memory_manager.go:365] "RemoveStaleState removing state" podUID="6719a2ae-c55d-477d-8853-f032599f9d45" containerName="cleanup"
Oct 30 13:15:29.382015 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:29.381871    4943 memory_manager.go:365] "RemoveStaleState removing state" podUID="bd5de1ce-17cd-4e87-bbac-766e14841a08" containerName="cleanup"
Oct 30 13:15:29.387191 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:29.387175    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-843da301-npccp"
Oct 30 13:15:29.426291 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:29.419208    4943 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-p9z57\""
Oct 30 13:15:29.426291 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:29.419249    4943 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 13:15:29.426291 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:29.424788    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-843da301-npccp"]
Oct 30 13:15:29.523608 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:29.523554    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/8464dcdb-97a0-4d19-a4ef-1d4e637f1738-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-843da301-npccp\" (UID: \"8464dcdb-97a0-4d19-a4ef-1d4e637f1738\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-843da301-npccp"
Oct 30 13:15:29.523608 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:29.523612    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fj6dh\" (UniqueName: \"kubernetes.io/projected/8464dcdb-97a0-4d19-a4ef-1d4e637f1738-kube-api-access-fj6dh\") pod \"maas-api-revoke-sub-843da301-npccp\" (UID: \"8464dcdb-97a0-4d19-a4ef-1d4e637f1738\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-843da301-npccp"
Oct 30 13:15:29.523888 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:29.523731    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/8464dcdb-97a0-4d19-a4ef-1d4e637f1738-maas-api-xks-ca\") pod \"maas-api-revoke-sub-843da301-npccp\" (UID: \"8464dcdb-97a0-4d19-a4ef-1d4e637f1738\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-843da301-npccp"
Oct 30 13:15:29.624745 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:29.624710    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/8464dcdb-97a0-4d19-a4ef-1d4e637f1738-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-843da301-npccp\" (UID: \"8464dcdb-97a0-4d19-a4ef-1d4e637f1738\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-843da301-npccp"
Oct 30 13:15:29.624921 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:29.624756    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fj6dh\" (UniqueName: \"kubernetes.io/projected/8464dcdb-97a0-4d19-a4ef-1d4e637f1738-kube-api-access-fj6dh\") pod \"maas-api-revoke-sub-843da301-npccp\" (UID: \"8464dcdb-97a0-4d19-a4ef-1d4e637f1738\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-843da301-npccp"
Oct 30 13:15:29.624921 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:29.624795    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/8464dcdb-97a0-4d19-a4ef-1d4e637f1738-maas-api-xks-ca\") pod \"maas-api-revoke-sub-843da301-npccp\" (UID: \"8464dcdb-97a0-4d19-a4ef-1d4e637f1738\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-843da301-npccp"
Oct 30 13:15:29.625416 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:29.625391    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/8464dcdb-97a0-4d19-a4ef-1d4e637f1738-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-843da301-npccp\" (UID: \"8464dcdb-97a0-4d19-a4ef-1d4e637f1738\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-843da301-npccp"
Oct 30 13:15:29.627059 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:29.627037    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/8464dcdb-97a0-4d19-a4ef-1d4e637f1738-maas-api-xks-ca\") pod \"maas-api-revoke-sub-843da301-npccp\" (UID: \"8464dcdb-97a0-4d19-a4ef-1d4e637f1738\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-843da301-npccp"
Oct 30 13:15:29.650658 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:29.650588    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fj6dh\" (UniqueName: \"kubernetes.io/projected/8464dcdb-97a0-4d19-a4ef-1d4e637f1738-kube-api-access-fj6dh\") pod \"maas-api-revoke-sub-843da301-npccp\" (UID: \"8464dcdb-97a0-4d19-a4ef-1d4e637f1738\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-843da301-npccp"
Oct 30 13:15:29.700488 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:29.700454    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-843da301-npccp"
Oct 30 13:15:29.984088 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:29.984045    4943 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-f55w2"
Oct 30 13:15:29.984826 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:29.984775    4943 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-f55w2"
Oct 30 13:15:29.986973 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:29.986947    4943 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-f55w2"
Oct 30 13:15:30.020089 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:30.020066    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-843da301-npccp"]
Oct 30 13:15:30.021667 ip-10-0-142-84 kubenswrapper[4943]: W1030 13:15:30.021641    4943 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8464dcdb_97a0_4d19_a4ef_1d4e637f1738.slice/crio-83c360c4a097a76b1c2d4ec4f2bef01b33d03aa7f19f8d219009569470b0c7fd WatchSource:0}: Error finding container 83c360c4a097a76b1c2d4ec4f2bef01b33d03aa7f19f8d219009569470b0c7fd: Status 404 returned error can't find the container with id 83c360c4a097a76b1c2d4ec4f2bef01b33d03aa7f19f8d219009569470b0c7fd
Oct 30 13:15:30.654620 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:30.654535    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-843da301-npccp" event={"ID":"8464dcdb-97a0-4d19-a4ef-1d4e637f1738","Type":"ContainerStarted","Data":"a8094c9252e39be8e23abff211fa0dc08678a24fefd1f5bb86af5ae8e43f765a"}
Oct 30 13:15:30.654620 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:30.654598    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-843da301-npccp" event={"ID":"8464dcdb-97a0-4d19-a4ef-1d4e637f1738","Type":"ContainerStarted","Data":"83c360c4a097a76b1c2d4ec4f2bef01b33d03aa7f19f8d219009569470b0c7fd"}
Oct 30 13:15:30.709785 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:30.709724    4943 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-843da301-npccp" podStartSLOduration=1.709710411 podStartE2EDuration="1.709710411s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:15:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:15:30.705857832 +0000 UTC m=+1106.493861562" watchObservedRunningTime="2026-10-30 13:15:30.709710411 +0000 UTC m=+1106.497714140"
Oct 30 13:15:31.662489 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:31.662453    4943 generic.go:356] "Generic (PLEG): container finished" podID="8464dcdb-97a0-4d19-a4ef-1d4e637f1738" containerID="a8094c9252e39be8e23abff211fa0dc08678a24fefd1f5bb86af5ae8e43f765a" exitCode=0
Oct 30 13:15:31.662986 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:31.662535    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-843da301-npccp" event={"ID":"8464dcdb-97a0-4d19-a4ef-1d4e637f1738","Type":"ContainerDied","Data":"a8094c9252e39be8e23abff211fa0dc08678a24fefd1f5bb86af5ae8e43f765a"}
Oct 30 13:15:32.852585 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:32.852549    4943 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-843da301-npccp"
Oct 30 13:15:32.969162 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:32.969128    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/8464dcdb-97a0-4d19-a4ef-1d4e637f1738-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/8464dcdb-97a0-4d19-a4ef-1d4e637f1738-maas-api-xks-ca\") pod \"8464dcdb-97a0-4d19-a4ef-1d4e637f1738\" (UID: \"8464dcdb-97a0-4d19-a4ef-1d4e637f1738\") "
Oct 30 13:15:32.969383 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:32.969196    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8464dcdb-97a0-4d19-a4ef-1d4e637f1738-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/8464dcdb-97a0-4d19-a4ef-1d4e637f1738-maas-api-openshift-ca\") pod \"8464dcdb-97a0-4d19-a4ef-1d4e637f1738\" (UID: \"8464dcdb-97a0-4d19-a4ef-1d4e637f1738\") "
Oct 30 13:15:32.969383 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:32.969277    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8464dcdb-97a0-4d19-a4ef-1d4e637f1738-kube-api-access-fj6dh\" (UniqueName: \"kubernetes.io/projected/8464dcdb-97a0-4d19-a4ef-1d4e637f1738-kube-api-access-fj6dh\") pod \"8464dcdb-97a0-4d19-a4ef-1d4e637f1738\" (UID: \"8464dcdb-97a0-4d19-a4ef-1d4e637f1738\") "
Oct 30 13:15:32.969913 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:32.969883    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8464dcdb-97a0-4d19-a4ef-1d4e637f1738-maas-api-openshift-ca" pod "8464dcdb-97a0-4d19-a4ef-1d4e637f1738" (UID: "8464dcdb-97a0-4d19-a4ef-1d4e637f1738"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 13:15:32.972412 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:32.971844    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8464dcdb-97a0-4d19-a4ef-1d4e637f1738-maas-api-xks-ca" pod "8464dcdb-97a0-4d19-a4ef-1d4e637f1738" (UID: "8464dcdb-97a0-4d19-a4ef-1d4e637f1738"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 13:15:32.974549 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:32.974526    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8464dcdb-97a0-4d19-a4ef-1d4e637f1738-kube-api-access-fj6dh" pod "8464dcdb-97a0-4d19-a4ef-1d4e637f1738" (UID: "8464dcdb-97a0-4d19-a4ef-1d4e637f1738"). InnerVolumeSpecName "kube-api-access-fj6dh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:15:33.070992 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:33.070955    4943 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/8464dcdb-97a0-4d19-a4ef-1d4e637f1738-maas-api-xks-ca\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:15:33.070992 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:33.070991    4943 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/8464dcdb-97a0-4d19-a4ef-1d4e637f1738-maas-api-openshift-ca\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:15:33.071185 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:33.071006    4943 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fj6dh\" (UniqueName: \"kubernetes.io/projected/8464dcdb-97a0-4d19-a4ef-1d4e637f1738-kube-api-access-fj6dh\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:15:33.683471 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:33.683440    4943 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-843da301-npccp"
Oct 30 13:15:33.683618 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:33.683485    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-843da301-npccp" event={"ID":"8464dcdb-97a0-4d19-a4ef-1d4e637f1738","Type":"ContainerDied","Data":"83c360c4a097a76b1c2d4ec4f2bef01b33d03aa7f19f8d219009569470b0c7fd"}
Oct 30 13:15:33.683618 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:33.683533    4943 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="83c360c4a097a76b1c2d4ec4f2bef01b33d03aa7f19f8d219009569470b0c7fd"
Oct 30 13:15:35.955854 ip-10-0-142-84 kubenswrapper[4943]: E1030 13:15:35.955822    4943 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-knqz6" containerName="extractor"
Oct 30 13:15:36.367618 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:36.367546    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-hdmrm"]
Oct 30 13:15:40.733924 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:40.733877    4943 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-1d7a6b59-lxwjg"]
Oct 30 13:15:40.735015 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:40.734987    4943 cpu_manager.go:414] "containerMap: removing container" podUID="fdaf8439-65b5-4e9a-b519-83d40aaf7307" containerName="cleanup"
Oct 30 13:15:40.735015 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:40.735012    4943 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fdaf8439-65b5-4e9a-b519-83d40aaf7307" containerName="cleanup"
Oct 30 13:15:40.735217 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:40.735042    4943 cpu_manager.go:414] "containerMap: removing container" podUID="8ec3dc68-1993-4289-bf82-f4e629c4d7ce" containerName="cleanup"
Oct 30 13:15:40.735217 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:40.735050    4943 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8ec3dc68-1993-4289-bf82-f4e629c4d7ce" containerName="cleanup"
Oct 30 13:15:40.735217 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:40.735060    4943 cpu_manager.go:414] "containerMap: removing container" podUID="6719a2ae-c55d-477d-8853-f032599f9d45" containerName="cleanup"
Oct 30 13:15:40.735217 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:40.735068    4943 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6719a2ae-c55d-477d-8853-f032599f9d45" containerName="cleanup"
Oct 30 13:15:40.735217 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:40.735112    4943 cpu_manager.go:414] "containerMap: removing container" podUID="8464dcdb-97a0-4d19-a4ef-1d4e637f1738" containerName="revoke-keys"
Oct 30 13:15:40.735217 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:40.735119    4943 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8464dcdb-97a0-4d19-a4ef-1d4e637f1738" containerName="revoke-keys"
Oct 30 13:15:40.735627 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:40.735242    4943 memory_manager.go:365] "RemoveStaleState removing state" podUID="8ec3dc68-1993-4289-bf82-f4e629c4d7ce" containerName="cleanup"
Oct 30 13:15:40.735627 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:40.735272    4943 memory_manager.go:365] "RemoveStaleState removing state" podUID="6719a2ae-c55d-477d-8853-f032599f9d45" containerName="cleanup"
Oct 30 13:15:40.735627 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:40.735289    4943 memory_manager.go:365] "RemoveStaleState removing state" podUID="fdaf8439-65b5-4e9a-b519-83d40aaf7307" containerName="cleanup"
Oct 30 13:15:40.735627 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:40.735297    4943 memory_manager.go:365] "RemoveStaleState removing state" podUID="8464dcdb-97a0-4d19-a4ef-1d4e637f1738" containerName="revoke-keys"
Oct 30 13:15:40.735627 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:40.735306    4943 memory_manager.go:365] "RemoveStaleState removing state" podUID="bd5de1ce-17cd-4e87-bbac-766e14841a08" containerName="cleanup"
Oct 30 13:15:40.740303 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:40.740254    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-1d7a6b59-lxwjg"
Oct 30 13:15:40.759547 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:40.759525    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-1d7a6b59-lxwjg"]
Oct 30 13:15:40.761765 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:40.761735    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2sr9p\" (UniqueName: \"kubernetes.io/projected/14a99554-9231-4607-b677-692c441093b0-kube-api-access-2sr9p\") pod \"maas-api-revoke-sub-1d7a6b59-lxwjg\" (UID: \"14a99554-9231-4607-b677-692c441093b0\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-1d7a6b59-lxwjg"
Oct 30 13:15:40.761973 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:40.761945    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/14a99554-9231-4607-b677-692c441093b0-maas-api-xks-ca\") pod \"maas-api-revoke-sub-1d7a6b59-lxwjg\" (UID: \"14a99554-9231-4607-b677-692c441093b0\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-1d7a6b59-lxwjg"
Oct 30 13:15:40.762051 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:40.762011    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/14a99554-9231-4607-b677-692c441093b0-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-1d7a6b59-lxwjg\" (UID: \"14a99554-9231-4607-b677-692c441093b0\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-1d7a6b59-lxwjg"
Oct 30 13:15:40.781711 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:40.768831    4943 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 13:15:40.781711 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:40.768843    4943 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-p9z57\""
Oct 30 13:15:40.863180 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:40.863148    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/14a99554-9231-4607-b677-692c441093b0-maas-api-xks-ca\") pod \"maas-api-revoke-sub-1d7a6b59-lxwjg\" (UID: \"14a99554-9231-4607-b677-692c441093b0\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-1d7a6b59-lxwjg"
Oct 30 13:15:40.863366 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:40.863201    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/14a99554-9231-4607-b677-692c441093b0-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-1d7a6b59-lxwjg\" (UID: \"14a99554-9231-4607-b677-692c441093b0\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-1d7a6b59-lxwjg"
Oct 30 13:15:40.863366 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:40.863297    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2sr9p\" (UniqueName: \"kubernetes.io/projected/14a99554-9231-4607-b677-692c441093b0-kube-api-access-2sr9p\") pod \"maas-api-revoke-sub-1d7a6b59-lxwjg\" (UID: \"14a99554-9231-4607-b677-692c441093b0\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-1d7a6b59-lxwjg"
Oct 30 13:15:40.864103 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:40.864074    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/14a99554-9231-4607-b677-692c441093b0-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-1d7a6b59-lxwjg\" (UID: \"14a99554-9231-4607-b677-692c441093b0\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-1d7a6b59-lxwjg"
Oct 30 13:15:40.865723 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:40.865690    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/14a99554-9231-4607-b677-692c441093b0-maas-api-xks-ca\") pod \"maas-api-revoke-sub-1d7a6b59-lxwjg\" (UID: \"14a99554-9231-4607-b677-692c441093b0\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-1d7a6b59-lxwjg"
Oct 30 13:15:40.888870 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:40.888840    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2sr9p\" (UniqueName: \"kubernetes.io/projected/14a99554-9231-4607-b677-692c441093b0-kube-api-access-2sr9p\") pod \"maas-api-revoke-sub-1d7a6b59-lxwjg\" (UID: \"14a99554-9231-4607-b677-692c441093b0\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-1d7a6b59-lxwjg"
Oct 30 13:15:41.054142 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:41.054061    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-1d7a6b59-lxwjg"
Oct 30 13:15:41.370738 ip-10-0-142-84 kubenswrapper[4943]: W1030 13:15:41.370707    4943 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod14a99554_9231_4607_b677_692c441093b0.slice/crio-3fdd98a1349afcf4dbe11975dfa68ea5ace329cf8f685a13a092c71cb2efddac WatchSource:0}: Error finding container 3fdd98a1349afcf4dbe11975dfa68ea5ace329cf8f685a13a092c71cb2efddac: Status 404 returned error can't find the container with id 3fdd98a1349afcf4dbe11975dfa68ea5ace329cf8f685a13a092c71cb2efddac
Oct 30 13:15:41.371054 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:41.371030    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-1d7a6b59-lxwjg"]
Oct 30 13:15:41.789292 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:41.789229    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-1d7a6b59-lxwjg" event={"ID":"14a99554-9231-4607-b677-692c441093b0","Type":"ContainerStarted","Data":"59aae9c206b0c61474afc19f14c1cb183b0001a2e0defd361fcf50552e93c494"}
Oct 30 13:15:41.789662 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:41.789309    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-1d7a6b59-lxwjg" event={"ID":"14a99554-9231-4607-b677-692c441093b0","Type":"ContainerStarted","Data":"3fdd98a1349afcf4dbe11975dfa68ea5ace329cf8f685a13a092c71cb2efddac"}
Oct 30 13:15:41.836389 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:41.836331    4943 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-1d7a6b59-lxwjg" podStartSLOduration=1.836309103 podStartE2EDuration="1.836309103s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:15:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:15:41.836036423 +0000 UTC m=+1117.624040154" watchObservedRunningTime="2026-10-30 13:15:41.836309103 +0000 UTC m=+1117.624312835"
Oct 30 13:15:42.798433 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:42.798398    4943 generic.go:356] "Generic (PLEG): container finished" podID="14a99554-9231-4607-b677-692c441093b0" containerID="59aae9c206b0c61474afc19f14c1cb183b0001a2e0defd361fcf50552e93c494" exitCode=0
Oct 30 13:15:42.798917 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:42.798506    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-1d7a6b59-lxwjg" event={"ID":"14a99554-9231-4607-b677-692c441093b0","Type":"ContainerDied","Data":"59aae9c206b0c61474afc19f14c1cb183b0001a2e0defd361fcf50552e93c494"}
Oct 30 13:15:43.240865 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:43.240831    4943 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-66cf4d45-xxxx2"]
Oct 30 13:15:43.245220 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:43.245197    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-66cf4d45-xxxx2"
Oct 30 13:15:43.267543 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:43.267515    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-66cf4d45-xxxx2"]
Oct 30 13:15:43.290271 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:43.290228    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/606ec1b4-0953-4cfc-a7e9-ea7832a39f11-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-66cf4d45-xxxx2\" (UID: \"606ec1b4-0953-4cfc-a7e9-ea7832a39f11\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-66cf4d45-xxxx2"
Oct 30 13:15:43.290399 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:43.290360    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/606ec1b4-0953-4cfc-a7e9-ea7832a39f11-maas-api-xks-ca\") pod \"maas-api-revoke-sub-66cf4d45-xxxx2\" (UID: \"606ec1b4-0953-4cfc-a7e9-ea7832a39f11\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-66cf4d45-xxxx2"
Oct 30 13:15:43.290479 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:43.290411    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-drrt5\" (UniqueName: \"kubernetes.io/projected/606ec1b4-0953-4cfc-a7e9-ea7832a39f11-kube-api-access-drrt5\") pod \"maas-api-revoke-sub-66cf4d45-xxxx2\" (UID: \"606ec1b4-0953-4cfc-a7e9-ea7832a39f11\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-66cf4d45-xxxx2"
Oct 30 13:15:43.391782 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:43.391744    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/606ec1b4-0953-4cfc-a7e9-ea7832a39f11-maas-api-xks-ca\") pod \"maas-api-revoke-sub-66cf4d45-xxxx2\" (UID: \"606ec1b4-0953-4cfc-a7e9-ea7832a39f11\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-66cf4d45-xxxx2"
Oct 30 13:15:43.391971 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:43.391794    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-drrt5\" (UniqueName: \"kubernetes.io/projected/606ec1b4-0953-4cfc-a7e9-ea7832a39f11-kube-api-access-drrt5\") pod \"maas-api-revoke-sub-66cf4d45-xxxx2\" (UID: \"606ec1b4-0953-4cfc-a7e9-ea7832a39f11\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-66cf4d45-xxxx2"
Oct 30 13:15:43.391971 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:43.391922    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/606ec1b4-0953-4cfc-a7e9-ea7832a39f11-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-66cf4d45-xxxx2\" (UID: \"606ec1b4-0953-4cfc-a7e9-ea7832a39f11\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-66cf4d45-xxxx2"
Oct 30 13:15:43.392635 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:43.392612    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/606ec1b4-0953-4cfc-a7e9-ea7832a39f11-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-66cf4d45-xxxx2\" (UID: \"606ec1b4-0953-4cfc-a7e9-ea7832a39f11\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-66cf4d45-xxxx2"
Oct 30 13:15:43.394112 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:43.394090    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/606ec1b4-0953-4cfc-a7e9-ea7832a39f11-maas-api-xks-ca\") pod \"maas-api-revoke-sub-66cf4d45-xxxx2\" (UID: \"606ec1b4-0953-4cfc-a7e9-ea7832a39f11\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-66cf4d45-xxxx2"
Oct 30 13:15:43.419437 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:43.419408    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-drrt5\" (UniqueName: \"kubernetes.io/projected/606ec1b4-0953-4cfc-a7e9-ea7832a39f11-kube-api-access-drrt5\") pod \"maas-api-revoke-sub-66cf4d45-xxxx2\" (UID: \"606ec1b4-0953-4cfc-a7e9-ea7832a39f11\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-66cf4d45-xxxx2"
Oct 30 13:15:43.559023 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:43.558947    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-66cf4d45-xxxx2"
Oct 30 13:15:43.880117 ip-10-0-142-84 kubenswrapper[4943]: W1030 13:15:43.880059    4943 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod606ec1b4_0953_4cfc_a7e9_ea7832a39f11.slice/crio-a7cc24e8a2e1f167e513cc3dcc12f7a6be58f43ef075038487c44f63a2afe622 WatchSource:0}: Error finding container a7cc24e8a2e1f167e513cc3dcc12f7a6be58f43ef075038487c44f63a2afe622: Status 404 returned error can't find the container with id a7cc24e8a2e1f167e513cc3dcc12f7a6be58f43ef075038487c44f63a2afe622
Oct 30 13:15:43.880579 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:43.880207    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-66cf4d45-xxxx2"]
Oct 30 13:15:43.983979 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:43.983954    4943 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-1d7a6b59-lxwjg"
Oct 30 13:15:43.997774 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:43.997750    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/14a99554-9231-4607-b677-692c441093b0-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/14a99554-9231-4607-b677-692c441093b0-maas-api-xks-ca\") pod \"14a99554-9231-4607-b677-692c441093b0\" (UID: \"14a99554-9231-4607-b677-692c441093b0\") "
Oct 30 13:15:43.997893 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:43.997828    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/14a99554-9231-4607-b677-692c441093b0-kube-api-access-2sr9p\" (UniqueName: \"kubernetes.io/projected/14a99554-9231-4607-b677-692c441093b0-kube-api-access-2sr9p\") pod \"14a99554-9231-4607-b677-692c441093b0\" (UID: \"14a99554-9231-4607-b677-692c441093b0\") "
Oct 30 13:15:43.997960 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:43.997948    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/14a99554-9231-4607-b677-692c441093b0-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/14a99554-9231-4607-b677-692c441093b0-maas-api-openshift-ca\") pod \"14a99554-9231-4607-b677-692c441093b0\" (UID: \"14a99554-9231-4607-b677-692c441093b0\") "
Oct 30 13:15:43.998307 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:43.998281    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/14a99554-9231-4607-b677-692c441093b0-maas-api-openshift-ca" pod "14a99554-9231-4607-b677-692c441093b0" (UID: "14a99554-9231-4607-b677-692c441093b0"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 13:15:43.999678 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:43.999657    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/14a99554-9231-4607-b677-692c441093b0-maas-api-xks-ca" pod "14a99554-9231-4607-b677-692c441093b0" (UID: "14a99554-9231-4607-b677-692c441093b0"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 13:15:43.999798 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:43.999778    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/14a99554-9231-4607-b677-692c441093b0-kube-api-access-2sr9p" pod "14a99554-9231-4607-b677-692c441093b0" (UID: "14a99554-9231-4607-b677-692c441093b0"). InnerVolumeSpecName "kube-api-access-2sr9p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:15:44.099756 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:44.099723    4943 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/14a99554-9231-4607-b677-692c441093b0-maas-api-xks-ca\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:15:44.099756 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:44.099753    4943 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2sr9p\" (UniqueName: \"kubernetes.io/projected/14a99554-9231-4607-b677-692c441093b0-kube-api-access-2sr9p\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:15:44.099960 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:44.099763    4943 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/14a99554-9231-4607-b677-692c441093b0-maas-api-openshift-ca\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:15:44.815301 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:44.815250    4943 generic.go:356] "Generic (PLEG): container finished" podID="606ec1b4-0953-4cfc-a7e9-ea7832a39f11" containerID="03b70ab6c2da08562cd652fbdaf1ead3503d608089d8bb67dcac1a69606a9588" exitCode=0
Oct 30 13:15:44.815301 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:44.815289    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-66cf4d45-xxxx2" event={"ID":"606ec1b4-0953-4cfc-a7e9-ea7832a39f11","Type":"ContainerDied","Data":"03b70ab6c2da08562cd652fbdaf1ead3503d608089d8bb67dcac1a69606a9588"}
Oct 30 13:15:44.815524 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:44.815341    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-66cf4d45-xxxx2" event={"ID":"606ec1b4-0953-4cfc-a7e9-ea7832a39f11","Type":"ContainerStarted","Data":"a7cc24e8a2e1f167e513cc3dcc12f7a6be58f43ef075038487c44f63a2afe622"}
Oct 30 13:15:44.816857 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:44.816834    4943 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-1d7a6b59-lxwjg"
Oct 30 13:15:44.816983 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:44.816864    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-1d7a6b59-lxwjg" event={"ID":"14a99554-9231-4607-b677-692c441093b0","Type":"ContainerDied","Data":"3fdd98a1349afcf4dbe11975dfa68ea5ace329cf8f685a13a092c71cb2efddac"}
Oct 30 13:15:44.816983 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:44.816913    4943 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3fdd98a1349afcf4dbe11975dfa68ea5ace329cf8f685a13a092c71cb2efddac"
Oct 30 13:15:46.003813 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:46.003787    4943 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-66cf4d45-xxxx2"
Oct 30 13:15:46.007562 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:46.007534    4943 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-kpvbd"]
Oct 30 13:15:46.008242 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:46.008219    4943 cpu_manager.go:414] "containerMap: removing container" podUID="14a99554-9231-4607-b677-692c441093b0" containerName="revoke-keys"
Oct 30 13:15:46.008242 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:46.008240    4943 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="14a99554-9231-4607-b677-692c441093b0" containerName="revoke-keys"
Oct 30 13:15:46.008392 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:46.008250    4943 cpu_manager.go:414] "containerMap: removing container" podUID="606ec1b4-0953-4cfc-a7e9-ea7832a39f11" containerName="revoke-keys"
Oct 30 13:15:46.008392 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:46.008255    4943 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="606ec1b4-0953-4cfc-a7e9-ea7832a39f11" containerName="revoke-keys"
Oct 30 13:15:46.008392 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:46.008363    4943 memory_manager.go:365] "RemoveStaleState removing state" podUID="14a99554-9231-4607-b677-692c441093b0" containerName="revoke-keys"
Oct 30 13:15:46.008392 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:46.008371    4943 memory_manager.go:365] "RemoveStaleState removing state" podUID="606ec1b4-0953-4cfc-a7e9-ea7832a39f11" containerName="revoke-keys"
Oct 30 13:15:46.012012 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:46.011992    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-kpvbd"
Oct 30 13:15:46.020423 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:46.020398    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/606ec1b4-0953-4cfc-a7e9-ea7832a39f11-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/606ec1b4-0953-4cfc-a7e9-ea7832a39f11-maas-api-openshift-ca\") pod \"606ec1b4-0953-4cfc-a7e9-ea7832a39f11\" (UID: \"606ec1b4-0953-4cfc-a7e9-ea7832a39f11\") "
Oct 30 13:15:46.020594 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:46.020578    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/606ec1b4-0953-4cfc-a7e9-ea7832a39f11-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/606ec1b4-0953-4cfc-a7e9-ea7832a39f11-maas-api-xks-ca\") pod \"606ec1b4-0953-4cfc-a7e9-ea7832a39f11\" (UID: \"606ec1b4-0953-4cfc-a7e9-ea7832a39f11\") "
Oct 30 13:15:46.020672 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:46.020655    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/606ec1b4-0953-4cfc-a7e9-ea7832a39f11-kube-api-access-drrt5\" (UniqueName: \"kubernetes.io/projected/606ec1b4-0953-4cfc-a7e9-ea7832a39f11-kube-api-access-drrt5\") pod \"606ec1b4-0953-4cfc-a7e9-ea7832a39f11\" (UID: \"606ec1b4-0953-4cfc-a7e9-ea7832a39f11\") "
Oct 30 13:15:46.020752 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:46.020729    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/606ec1b4-0953-4cfc-a7e9-ea7832a39f11-maas-api-openshift-ca" pod "606ec1b4-0953-4cfc-a7e9-ea7832a39f11" (UID: "606ec1b4-0953-4cfc-a7e9-ea7832a39f11"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 13:15:46.021044 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:46.021027    4943 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/606ec1b4-0953-4cfc-a7e9-ea7832a39f11-maas-api-openshift-ca\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:15:46.022708 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:46.022682    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/606ec1b4-0953-4cfc-a7e9-ea7832a39f11-maas-api-xks-ca" pod "606ec1b4-0953-4cfc-a7e9-ea7832a39f11" (UID: "606ec1b4-0953-4cfc-a7e9-ea7832a39f11"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 13:15:46.022811 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:46.022794    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/606ec1b4-0953-4cfc-a7e9-ea7832a39f11-kube-api-access-drrt5" pod "606ec1b4-0953-4cfc-a7e9-ea7832a39f11" (UID: "606ec1b4-0953-4cfc-a7e9-ea7832a39f11"). InnerVolumeSpecName "kube-api-access-drrt5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:15:46.056497 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:46.055985    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-kpvbd"]
Oct 30 13:15:46.121642 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:46.121556    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mhwd5\" (UniqueName: \"kubernetes.io/projected/f0e82596-0383-4ec9-a856-a6234e9da808-kube-api-access-mhwd5\") pod \"e2e-worker-w2-f76c09-openshift-default-65d5845b67-kpvbd\" (UID: \"f0e82596-0383-4ec9-a856-a6234e9da808\") " pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-kpvbd"
Oct 30 13:15:46.121642 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:46.121593    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f0e82596-0383-4ec9-a856-a6234e9da808-istio-data\") pod \"e2e-worker-w2-f76c09-openshift-default-65d5845b67-kpvbd\" (UID: \"f0e82596-0383-4ec9-a856-a6234e9da808\") " pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-kpvbd"
Oct 30 13:15:46.121642 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:46.121624    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f0e82596-0383-4ec9-a856-a6234e9da808-workload-socket\") pod \"e2e-worker-w2-f76c09-openshift-default-65d5845b67-kpvbd\" (UID: \"f0e82596-0383-4ec9-a856-a6234e9da808\") " pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-kpvbd"
Oct 30 13:15:46.121890 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:46.121670    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f0e82596-0383-4ec9-a856-a6234e9da808-credential-socket\") pod \"e2e-worker-w2-f76c09-openshift-default-65d5845b67-kpvbd\" (UID: \"f0e82596-0383-4ec9-a856-a6234e9da808\") " pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-kpvbd"
Oct 30 13:15:46.121890 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:46.121801    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f0e82596-0383-4ec9-a856-a6234e9da808-istio-token\") pod \"e2e-worker-w2-f76c09-openshift-default-65d5845b67-kpvbd\" (UID: \"f0e82596-0383-4ec9-a856-a6234e9da808\") " pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-kpvbd"
Oct 30 13:15:46.121890 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:46.121868    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f0e82596-0383-4ec9-a856-a6234e9da808-workload-certs\") pod \"e2e-worker-w2-f76c09-openshift-default-65d5845b67-kpvbd\" (UID: \"f0e82596-0383-4ec9-a856-a6234e9da808\") " pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-kpvbd"
Oct 30 13:15:46.122016 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:46.121909    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f0e82596-0383-4ec9-a856-a6234e9da808-istiod-ca-cert\") pod \"e2e-worker-w2-f76c09-openshift-default-65d5845b67-kpvbd\" (UID: \"f0e82596-0383-4ec9-a856-a6234e9da808\") " pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-kpvbd"
Oct 30 13:15:46.122016 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:46.121944    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f0e82596-0383-4ec9-a856-a6234e9da808-istio-envoy\") pod \"e2e-worker-w2-f76c09-openshift-default-65d5845b67-kpvbd\" (UID: \"f0e82596-0383-4ec9-a856-a6234e9da808\") " pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-kpvbd"
Oct 30 13:15:46.122016 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:46.121963    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f0e82596-0383-4ec9-a856-a6234e9da808-istio-podinfo\") pod \"e2e-worker-w2-f76c09-openshift-default-65d5845b67-kpvbd\" (UID: \"f0e82596-0383-4ec9-a856-a6234e9da808\") " pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-kpvbd"
Oct 30 13:15:46.122120 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:46.122087    4943 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/606ec1b4-0953-4cfc-a7e9-ea7832a39f11-maas-api-xks-ca\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:15:46.122120 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:46.122101    4943 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-drrt5\" (UniqueName: \"kubernetes.io/projected/606ec1b4-0953-4cfc-a7e9-ea7832a39f11-kube-api-access-drrt5\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:15:46.230885 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:46.230835    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mhwd5\" (UniqueName: \"kubernetes.io/projected/f0e82596-0383-4ec9-a856-a6234e9da808-kube-api-access-mhwd5\") pod \"e2e-worker-w2-f76c09-openshift-default-65d5845b67-kpvbd\" (UID: \"f0e82596-0383-4ec9-a856-a6234e9da808\") " pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-kpvbd"
Oct 30 13:15:46.231129 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:46.231109    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f0e82596-0383-4ec9-a856-a6234e9da808-istio-data\") pod \"e2e-worker-w2-f76c09-openshift-default-65d5845b67-kpvbd\" (UID: \"f0e82596-0383-4ec9-a856-a6234e9da808\") " pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-kpvbd"
Oct 30 13:15:46.231322 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:46.231303    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f0e82596-0383-4ec9-a856-a6234e9da808-workload-socket\") pod \"e2e-worker-w2-f76c09-openshift-default-65d5845b67-kpvbd\" (UID: \"f0e82596-0383-4ec9-a856-a6234e9da808\") " pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-kpvbd"
Oct 30 13:15:46.231446 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:46.231432    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f0e82596-0383-4ec9-a856-a6234e9da808-credential-socket\") pod \"e2e-worker-w2-f76c09-openshift-default-65d5845b67-kpvbd\" (UID: \"f0e82596-0383-4ec9-a856-a6234e9da808\") " pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-kpvbd"
Oct 30 13:15:46.231563 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:46.231550    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f0e82596-0383-4ec9-a856-a6234e9da808-istio-token\") pod \"e2e-worker-w2-f76c09-openshift-default-65d5845b67-kpvbd\" (UID: \"f0e82596-0383-4ec9-a856-a6234e9da808\") " pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-kpvbd"
Oct 30 13:15:46.231683 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:46.231669    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f0e82596-0383-4ec9-a856-a6234e9da808-workload-certs\") pod \"e2e-worker-w2-f76c09-openshift-default-65d5845b67-kpvbd\" (UID: \"f0e82596-0383-4ec9-a856-a6234e9da808\") " pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-kpvbd"
Oct 30 13:15:46.231801 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:46.231787    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f0e82596-0383-4ec9-a856-a6234e9da808-istiod-ca-cert\") pod \"e2e-worker-w2-f76c09-openshift-default-65d5845b67-kpvbd\" (UID: \"f0e82596-0383-4ec9-a856-a6234e9da808\") " pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-kpvbd"
Oct 30 13:15:46.231907 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:46.231894    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f0e82596-0383-4ec9-a856-a6234e9da808-istio-envoy\") pod \"e2e-worker-w2-f76c09-openshift-default-65d5845b67-kpvbd\" (UID: \"f0e82596-0383-4ec9-a856-a6234e9da808\") " pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-kpvbd"
Oct 30 13:15:46.232005 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:46.231991    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f0e82596-0383-4ec9-a856-a6234e9da808-istio-podinfo\") pod \"e2e-worker-w2-f76c09-openshift-default-65d5845b67-kpvbd\" (UID: \"f0e82596-0383-4ec9-a856-a6234e9da808\") " pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-kpvbd"
Oct 30 13:15:46.233387 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:46.233354    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f0e82596-0383-4ec9-a856-a6234e9da808-credential-socket\") pod \"e2e-worker-w2-f76c09-openshift-default-65d5845b67-kpvbd\" (UID: \"f0e82596-0383-4ec9-a856-a6234e9da808\") " pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-kpvbd"
Oct 30 13:15:46.233537 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:46.233481    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f0e82596-0383-4ec9-a856-a6234e9da808-istio-data\") pod \"e2e-worker-w2-f76c09-openshift-default-65d5845b67-kpvbd\" (UID: \"f0e82596-0383-4ec9-a856-a6234e9da808\") " pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-kpvbd"
Oct 30 13:15:46.233633 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:46.233609    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f0e82596-0383-4ec9-a856-a6234e9da808-workload-socket\") pod \"e2e-worker-w2-f76c09-openshift-default-65d5845b67-kpvbd\" (UID: \"f0e82596-0383-4ec9-a856-a6234e9da808\") " pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-kpvbd"
Oct 30 13:15:46.233738 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:46.233711    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f0e82596-0383-4ec9-a856-a6234e9da808-istiod-ca-cert\") pod \"e2e-worker-w2-f76c09-openshift-default-65d5845b67-kpvbd\" (UID: \"f0e82596-0383-4ec9-a856-a6234e9da808\") " pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-kpvbd"
Oct 30 13:15:46.233935 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:46.233717    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f0e82596-0383-4ec9-a856-a6234e9da808-workload-certs\") pod \"e2e-worker-w2-f76c09-openshift-default-65d5845b67-kpvbd\" (UID: \"f0e82596-0383-4ec9-a856-a6234e9da808\") " pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-kpvbd"
Oct 30 13:15:46.235964 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:46.235944    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f0e82596-0383-4ec9-a856-a6234e9da808-istio-podinfo\") pod \"e2e-worker-w2-f76c09-openshift-default-65d5845b67-kpvbd\" (UID: \"f0e82596-0383-4ec9-a856-a6234e9da808\") " pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-kpvbd"
Oct 30 13:15:46.236051 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:46.236026    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f0e82596-0383-4ec9-a856-a6234e9da808-istio-envoy\") pod \"e2e-worker-w2-f76c09-openshift-default-65d5845b67-kpvbd\" (UID: \"f0e82596-0383-4ec9-a856-a6234e9da808\") " pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-kpvbd"
Oct 30 13:15:46.299283 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:46.299098    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mhwd5\" (UniqueName: \"kubernetes.io/projected/f0e82596-0383-4ec9-a856-a6234e9da808-kube-api-access-mhwd5\") pod \"e2e-worker-w2-f76c09-openshift-default-65d5845b67-kpvbd\" (UID: \"f0e82596-0383-4ec9-a856-a6234e9da808\") " pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-kpvbd"
Oct 30 13:15:46.301779 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:46.299998    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f0e82596-0383-4ec9-a856-a6234e9da808-istio-token\") pod \"e2e-worker-w2-f76c09-openshift-default-65d5845b67-kpvbd\" (UID: \"f0e82596-0383-4ec9-a856-a6234e9da808\") " pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-kpvbd"
Oct 30 13:15:46.328475 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:46.328440    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-kpvbd"
Oct 30 13:15:46.679980 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:46.679948    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-kpvbd"]
Oct 30 13:15:46.680541 ip-10-0-142-84 kubenswrapper[4943]: W1030 13:15:46.680512    4943 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf0e82596_0383_4ec9_a856_a6234e9da808.slice/crio-ed430c976530b0ea483a00e6f204a2c40d441a40cab4b6132652269f854fc246 WatchSource:0}: Error finding container ed430c976530b0ea483a00e6f204a2c40d441a40cab4b6132652269f854fc246: Status 404 returned error can't find the container with id ed430c976530b0ea483a00e6f204a2c40d441a40cab4b6132652269f854fc246
Oct 30 13:15:46.682998 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:46.682960    4943 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 13:15:46.683089 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:46.683028    4943 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 13:15:46.683089 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:46.683057    4943 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 13:15:46.835076 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:46.835046    4943 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-66cf4d45-xxxx2"
Oct 30 13:15:46.835076 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:46.835066    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-66cf4d45-xxxx2" event={"ID":"606ec1b4-0953-4cfc-a7e9-ea7832a39f11","Type":"ContainerDied","Data":"a7cc24e8a2e1f167e513cc3dcc12f7a6be58f43ef075038487c44f63a2afe622"}
Oct 30 13:15:46.835312 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:46.835115    4943 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a7cc24e8a2e1f167e513cc3dcc12f7a6be58f43ef075038487c44f63a2afe622"
Oct 30 13:15:46.837105 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:46.837075    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-kpvbd" event={"ID":"f0e82596-0383-4ec9-a856-a6234e9da808","Type":"ContainerStarted","Data":"0717bcb2ccdab96dd7a6937fd08a0fe5581f31152a2d124a5306480719d7c3f7"}
Oct 30 13:15:46.837247 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:46.837124    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-kpvbd" event={"ID":"f0e82596-0383-4ec9-a856-a6234e9da808","Type":"ContainerStarted","Data":"ed430c976530b0ea483a00e6f204a2c40d441a40cab4b6132652269f854fc246"}
Oct 30 13:15:46.850941 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:46.850913    4943 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-49abdeb7-g7kx7"]
Oct 30 13:15:46.855250 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:46.855228    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-49abdeb7-g7kx7"
Oct 30 13:15:46.876601 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:46.876575    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-49abdeb7-g7kx7"]
Oct 30 13:15:46.877350 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:46.877320    4943 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 13:15:46.877475 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:46.877452    4943 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-p9z57\""
Oct 30 13:15:46.891881 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:46.891810    4943 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-kpvbd" podStartSLOduration=1.8917922379999998 podStartE2EDuration="1.891792238s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:15:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:15:46.889392386 +0000 UTC m=+1122.677396116" watchObservedRunningTime="2026-10-30 13:15:46.891792238 +0000 UTC m=+1122.679795980"
Oct 30 13:15:46.941283 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:46.941180    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j7qjc\" (UniqueName: \"kubernetes.io/projected/ef06d03d-a788-4c5e-919c-c67521941405-kube-api-access-j7qjc\") pod \"maas-api-revoke-sub-49abdeb7-g7kx7\" (UID: \"ef06d03d-a788-4c5e-919c-c67521941405\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-49abdeb7-g7kx7"
Oct 30 13:15:46.941283 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:46.941242    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/ef06d03d-a788-4c5e-919c-c67521941405-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-49abdeb7-g7kx7\" (UID: \"ef06d03d-a788-4c5e-919c-c67521941405\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-49abdeb7-g7kx7"
Oct 30 13:15:46.941504 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:46.941475    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/ef06d03d-a788-4c5e-919c-c67521941405-maas-api-xks-ca\") pod \"maas-api-revoke-sub-49abdeb7-g7kx7\" (UID: \"ef06d03d-a788-4c5e-919c-c67521941405\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-49abdeb7-g7kx7"
Oct 30 13:15:47.042842 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:47.042804    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j7qjc\" (UniqueName: \"kubernetes.io/projected/ef06d03d-a788-4c5e-919c-c67521941405-kube-api-access-j7qjc\") pod \"maas-api-revoke-sub-49abdeb7-g7kx7\" (UID: \"ef06d03d-a788-4c5e-919c-c67521941405\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-49abdeb7-g7kx7"
Oct 30 13:15:47.043229 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:47.042850    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/ef06d03d-a788-4c5e-919c-c67521941405-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-49abdeb7-g7kx7\" (UID: \"ef06d03d-a788-4c5e-919c-c67521941405\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-49abdeb7-g7kx7"
Oct 30 13:15:47.043229 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:47.042896    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/ef06d03d-a788-4c5e-919c-c67521941405-maas-api-xks-ca\") pod \"maas-api-revoke-sub-49abdeb7-g7kx7\" (UID: \"ef06d03d-a788-4c5e-919c-c67521941405\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-49abdeb7-g7kx7"
Oct 30 13:15:47.043769 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:47.043744    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/ef06d03d-a788-4c5e-919c-c67521941405-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-49abdeb7-g7kx7\" (UID: \"ef06d03d-a788-4c5e-919c-c67521941405\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-49abdeb7-g7kx7"
Oct 30 13:15:47.045339 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:47.045315    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/ef06d03d-a788-4c5e-919c-c67521941405-maas-api-xks-ca\") pod \"maas-api-revoke-sub-49abdeb7-g7kx7\" (UID: \"ef06d03d-a788-4c5e-919c-c67521941405\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-49abdeb7-g7kx7"
Oct 30 13:15:47.090921 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:47.090884    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j7qjc\" (UniqueName: \"kubernetes.io/projected/ef06d03d-a788-4c5e-919c-c67521941405-kube-api-access-j7qjc\") pod \"maas-api-revoke-sub-49abdeb7-g7kx7\" (UID: \"ef06d03d-a788-4c5e-919c-c67521941405\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-49abdeb7-g7kx7"
Oct 30 13:15:47.171528 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:47.171474    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-49abdeb7-g7kx7"
Oct 30 13:15:47.329651 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:47.329606    4943 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-kpvbd"
Oct 30 13:15:47.330917 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:47.330889    4943 patch_prober.go:29] interesting pod/e2e-worker-w2-f76c09-openshift-default-65d5845b67-kpvbd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused" start-of-body=
Oct 30 13:15:47.331041 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:47.330937    4943 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-kpvbd" podUID="f0e82596-0383-4ec9-a856-a6234e9da808" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused"
Oct 30 13:15:47.492977 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:47.492948    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-49abdeb7-g7kx7"]
Oct 30 13:15:47.493715 ip-10-0-142-84 kubenswrapper[4943]: W1030 13:15:47.493687    4943 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef06d03d_a788_4c5e_919c_c67521941405.slice/crio-bba8d8aa07e9b0c54c03963841db96d28fc9beac80f49124ec571027e3d283d6 WatchSource:0}: Error finding container bba8d8aa07e9b0c54c03963841db96d28fc9beac80f49124ec571027e3d283d6: Status 404 returned error can't find the container with id bba8d8aa07e9b0c54c03963841db96d28fc9beac80f49124ec571027e3d283d6
Oct 30 13:15:47.845693 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:47.845604    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-49abdeb7-g7kx7" event={"ID":"ef06d03d-a788-4c5e-919c-c67521941405","Type":"ContainerStarted","Data":"bba8d8aa07e9b0c54c03963841db96d28fc9beac80f49124ec571027e3d283d6"}
Oct 30 13:15:48.329695 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:48.329658    4943 patch_prober.go:29] interesting pod/e2e-worker-w2-f76c09-openshift-default-65d5845b67-kpvbd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused" start-of-body=
Oct 30 13:15:48.330137 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:48.329715    4943 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-kpvbd" podUID="f0e82596-0383-4ec9-a856-a6234e9da808" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused"
Oct 30 13:15:48.855212 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:48.855171    4943 generic.go:356] "Generic (PLEG): container finished" podID="ef06d03d-a788-4c5e-919c-c67521941405" containerID="a637a84f7281a0834c1f26089f096423f32715fb8969c01ee7d193735716e06a" exitCode=0
Oct 30 13:15:48.855474 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:48.855288    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-49abdeb7-g7kx7" event={"ID":"ef06d03d-a788-4c5e-919c-c67521941405","Type":"ContainerDied","Data":"a637a84f7281a0834c1f26089f096423f32715fb8969c01ee7d193735716e06a"}
Oct 30 13:15:49.328959 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:49.328929    4943 patch_prober.go:29] interesting pod/e2e-worker-w2-f76c09-openshift-default-65d5845b67-kpvbd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused" start-of-body=
Oct 30 13:15:49.329195 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:49.329168    4943 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-kpvbd" podUID="f0e82596-0383-4ec9-a856-a6234e9da808" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused"
Oct 30 13:15:50.094375 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:50.094342    4943 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-49abdeb7-g7kx7"
Oct 30 13:15:50.209895 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:50.209859    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ef06d03d-a788-4c5e-919c-c67521941405-kube-api-access-j7qjc\" (UniqueName: \"kubernetes.io/projected/ef06d03d-a788-4c5e-919c-c67521941405-kube-api-access-j7qjc\") pod \"ef06d03d-a788-4c5e-919c-c67521941405\" (UID: \"ef06d03d-a788-4c5e-919c-c67521941405\") "
Oct 30 13:15:50.210086 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:50.209903    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/ef06d03d-a788-4c5e-919c-c67521941405-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/ef06d03d-a788-4c5e-919c-c67521941405-maas-api-xks-ca\") pod \"ef06d03d-a788-4c5e-919c-c67521941405\" (UID: \"ef06d03d-a788-4c5e-919c-c67521941405\") "
Oct 30 13:15:50.210086 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:50.209926    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ef06d03d-a788-4c5e-919c-c67521941405-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/ef06d03d-a788-4c5e-919c-c67521941405-maas-api-openshift-ca\") pod \"ef06d03d-a788-4c5e-919c-c67521941405\" (UID: \"ef06d03d-a788-4c5e-919c-c67521941405\") "
Oct 30 13:15:50.210512 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:50.210490    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ef06d03d-a788-4c5e-919c-c67521941405-maas-api-openshift-ca" pod "ef06d03d-a788-4c5e-919c-c67521941405" (UID: "ef06d03d-a788-4c5e-919c-c67521941405"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 13:15:50.211954 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:50.211930    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ef06d03d-a788-4c5e-919c-c67521941405-maas-api-xks-ca" pod "ef06d03d-a788-4c5e-919c-c67521941405" (UID: "ef06d03d-a788-4c5e-919c-c67521941405"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 13:15:50.212134 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:50.212110    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ef06d03d-a788-4c5e-919c-c67521941405-kube-api-access-j7qjc" pod "ef06d03d-a788-4c5e-919c-c67521941405" (UID: "ef06d03d-a788-4c5e-919c-c67521941405"). InnerVolumeSpecName "kube-api-access-j7qjc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:15:50.310879 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:50.310844    4943 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j7qjc\" (UniqueName: \"kubernetes.io/projected/ef06d03d-a788-4c5e-919c-c67521941405-kube-api-access-j7qjc\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:15:50.310879 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:50.310876    4943 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/ef06d03d-a788-4c5e-919c-c67521941405-maas-api-xks-ca\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:15:50.310879 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:50.310885    4943 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/ef06d03d-a788-4c5e-919c-c67521941405-maas-api-openshift-ca\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:15:50.329399 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:50.329364    4943 patch_prober.go:29] interesting pod/e2e-worker-w2-f76c09-openshift-default-65d5845b67-kpvbd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused" start-of-body=
Oct 30 13:15:50.329530 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:50.329413    4943 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-kpvbd" podUID="f0e82596-0383-4ec9-a856-a6234e9da808" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused"
Oct 30 13:15:50.827978 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:50.827941    4943 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-nbdn6"]
Oct 30 13:15:50.828531 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:50.828516    4943 cpu_manager.go:414] "containerMap: removing container" podUID="ef06d03d-a788-4c5e-919c-c67521941405" containerName="revoke-keys"
Oct 30 13:15:50.828595 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:50.828532    4943 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ef06d03d-a788-4c5e-919c-c67521941405" containerName="revoke-keys"
Oct 30 13:15:50.828664 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:50.828654    4943 memory_manager.go:365] "RemoveStaleState removing state" podUID="ef06d03d-a788-4c5e-919c-c67521941405" containerName="revoke-keys"
Oct 30 13:15:50.832216 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:50.832087    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-nbdn6"
Oct 30 13:15:50.892944 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:50.892911    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-nbdn6"]
Oct 30 13:15:50.901186 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:50.901160    4943 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-49abdeb7-g7kx7"
Oct 30 13:15:50.901388 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:50.901195    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-49abdeb7-g7kx7" event={"ID":"ef06d03d-a788-4c5e-919c-c67521941405","Type":"ContainerDied","Data":"bba8d8aa07e9b0c54c03963841db96d28fc9beac80f49124ec571027e3d283d6"}
Oct 30 13:15:50.901388 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:50.901243    4943 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="bba8d8aa07e9b0c54c03963841db96d28fc9beac80f49124ec571027e3d283d6"
Oct 30 13:15:50.917970 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:50.917946    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m2dgf\" (UniqueName: \"kubernetes.io/projected/7088cc53-0ec6-43be-9669-075ca0ad2fa1-kube-api-access-m2dgf\") pod \"e2e-worker-w0-031737-openshift-default-64b8f46f47-nbdn6\" (UID: \"7088cc53-0ec6-43be-9669-075ca0ad2fa1\") " pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-nbdn6"
Oct 30 13:15:50.918079 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:50.918052    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7088cc53-0ec6-43be-9669-075ca0ad2fa1-credential-socket\") pod \"e2e-worker-w0-031737-openshift-default-64b8f46f47-nbdn6\" (UID: \"7088cc53-0ec6-43be-9669-075ca0ad2fa1\") " pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-nbdn6"
Oct 30 13:15:50.918130 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:50.918090    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7088cc53-0ec6-43be-9669-075ca0ad2fa1-istio-data\") pod \"e2e-worker-w0-031737-openshift-default-64b8f46f47-nbdn6\" (UID: \"7088cc53-0ec6-43be-9669-075ca0ad2fa1\") " pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-nbdn6"
Oct 30 13:15:50.918188 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:50.918170    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7088cc53-0ec6-43be-9669-075ca0ad2fa1-istio-envoy\") pod \"e2e-worker-w0-031737-openshift-default-64b8f46f47-nbdn6\" (UID: \"7088cc53-0ec6-43be-9669-075ca0ad2fa1\") " pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-nbdn6"
Oct 30 13:15:50.918233 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:50.918210    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7088cc53-0ec6-43be-9669-075ca0ad2fa1-istio-podinfo\") pod \"e2e-worker-w0-031737-openshift-default-64b8f46f47-nbdn6\" (UID: \"7088cc53-0ec6-43be-9669-075ca0ad2fa1\") " pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-nbdn6"
Oct 30 13:15:50.918294 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:50.918273    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7088cc53-0ec6-43be-9669-075ca0ad2fa1-workload-socket\") pod \"e2e-worker-w0-031737-openshift-default-64b8f46f47-nbdn6\" (UID: \"7088cc53-0ec6-43be-9669-075ca0ad2fa1\") " pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-nbdn6"
Oct 30 13:15:50.918335 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:50.918314    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7088cc53-0ec6-43be-9669-075ca0ad2fa1-workload-certs\") pod \"e2e-worker-w0-031737-openshift-default-64b8f46f47-nbdn6\" (UID: \"7088cc53-0ec6-43be-9669-075ca0ad2fa1\") " pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-nbdn6"
Oct 30 13:15:50.918372 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:50.918333    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7088cc53-0ec6-43be-9669-075ca0ad2fa1-istio-token\") pod \"e2e-worker-w0-031737-openshift-default-64b8f46f47-nbdn6\" (UID: \"7088cc53-0ec6-43be-9669-075ca0ad2fa1\") " pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-nbdn6"
Oct 30 13:15:50.918407 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:50.918376    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7088cc53-0ec6-43be-9669-075ca0ad2fa1-istiod-ca-cert\") pod \"e2e-worker-w0-031737-openshift-default-64b8f46f47-nbdn6\" (UID: \"7088cc53-0ec6-43be-9669-075ca0ad2fa1\") " pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-nbdn6"
Oct 30 13:15:51.019986 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:51.019951    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m2dgf\" (UniqueName: \"kubernetes.io/projected/7088cc53-0ec6-43be-9669-075ca0ad2fa1-kube-api-access-m2dgf\") pod \"e2e-worker-w0-031737-openshift-default-64b8f46f47-nbdn6\" (UID: \"7088cc53-0ec6-43be-9669-075ca0ad2fa1\") " pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-nbdn6"
Oct 30 13:15:51.020174 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:51.020006    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7088cc53-0ec6-43be-9669-075ca0ad2fa1-credential-socket\") pod \"e2e-worker-w0-031737-openshift-default-64b8f46f47-nbdn6\" (UID: \"7088cc53-0ec6-43be-9669-075ca0ad2fa1\") " pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-nbdn6"
Oct 30 13:15:51.020174 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:51.020033    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7088cc53-0ec6-43be-9669-075ca0ad2fa1-istio-data\") pod \"e2e-worker-w0-031737-openshift-default-64b8f46f47-nbdn6\" (UID: \"7088cc53-0ec6-43be-9669-075ca0ad2fa1\") " pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-nbdn6"
Oct 30 13:15:51.020174 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:51.020057    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7088cc53-0ec6-43be-9669-075ca0ad2fa1-istio-envoy\") pod \"e2e-worker-w0-031737-openshift-default-64b8f46f47-nbdn6\" (UID: \"7088cc53-0ec6-43be-9669-075ca0ad2fa1\") " pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-nbdn6"
Oct 30 13:15:51.020174 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:51.020083    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7088cc53-0ec6-43be-9669-075ca0ad2fa1-istio-podinfo\") pod \"e2e-worker-w0-031737-openshift-default-64b8f46f47-nbdn6\" (UID: \"7088cc53-0ec6-43be-9669-075ca0ad2fa1\") " pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-nbdn6"
Oct 30 13:15:51.020174 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:51.020106    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7088cc53-0ec6-43be-9669-075ca0ad2fa1-workload-socket\") pod \"e2e-worker-w0-031737-openshift-default-64b8f46f47-nbdn6\" (UID: \"7088cc53-0ec6-43be-9669-075ca0ad2fa1\") " pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-nbdn6"
Oct 30 13:15:51.020174 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:51.020121    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7088cc53-0ec6-43be-9669-075ca0ad2fa1-workload-certs\") pod \"e2e-worker-w0-031737-openshift-default-64b8f46f47-nbdn6\" (UID: \"7088cc53-0ec6-43be-9669-075ca0ad2fa1\") " pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-nbdn6"
Oct 30 13:15:51.020174 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:51.020136    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7088cc53-0ec6-43be-9669-075ca0ad2fa1-istio-token\") pod \"e2e-worker-w0-031737-openshift-default-64b8f46f47-nbdn6\" (UID: \"7088cc53-0ec6-43be-9669-075ca0ad2fa1\") " pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-nbdn6"
Oct 30 13:15:51.020642 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:51.020179    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7088cc53-0ec6-43be-9669-075ca0ad2fa1-istiod-ca-cert\") pod \"e2e-worker-w0-031737-openshift-default-64b8f46f47-nbdn6\" (UID: \"7088cc53-0ec6-43be-9669-075ca0ad2fa1\") " pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-nbdn6"
Oct 30 13:15:51.020642 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:51.020579    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7088cc53-0ec6-43be-9669-075ca0ad2fa1-workload-certs\") pod \"e2e-worker-w0-031737-openshift-default-64b8f46f47-nbdn6\" (UID: \"7088cc53-0ec6-43be-9669-075ca0ad2fa1\") " pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-nbdn6"
Oct 30 13:15:51.020737 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:51.020653    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7088cc53-0ec6-43be-9669-075ca0ad2fa1-workload-socket\") pod \"e2e-worker-w0-031737-openshift-default-64b8f46f47-nbdn6\" (UID: \"7088cc53-0ec6-43be-9669-075ca0ad2fa1\") " pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-nbdn6"
Oct 30 13:15:51.020737 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:51.020713    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7088cc53-0ec6-43be-9669-075ca0ad2fa1-credential-socket\") pod \"e2e-worker-w0-031737-openshift-default-64b8f46f47-nbdn6\" (UID: \"7088cc53-0ec6-43be-9669-075ca0ad2fa1\") " pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-nbdn6"
Oct 30 13:15:51.020979 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:51.020959    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7088cc53-0ec6-43be-9669-075ca0ad2fa1-istio-data\") pod \"e2e-worker-w0-031737-openshift-default-64b8f46f47-nbdn6\" (UID: \"7088cc53-0ec6-43be-9669-075ca0ad2fa1\") " pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-nbdn6"
Oct 30 13:15:51.021445 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:51.021424    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7088cc53-0ec6-43be-9669-075ca0ad2fa1-istiod-ca-cert\") pod \"e2e-worker-w0-031737-openshift-default-64b8f46f47-nbdn6\" (UID: \"7088cc53-0ec6-43be-9669-075ca0ad2fa1\") " pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-nbdn6"
Oct 30 13:15:51.022459 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:51.022434    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7088cc53-0ec6-43be-9669-075ca0ad2fa1-istio-envoy\") pod \"e2e-worker-w0-031737-openshift-default-64b8f46f47-nbdn6\" (UID: \"7088cc53-0ec6-43be-9669-075ca0ad2fa1\") " pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-nbdn6"
Oct 30 13:15:51.024273 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:51.024240    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7088cc53-0ec6-43be-9669-075ca0ad2fa1-istio-podinfo\") pod \"e2e-worker-w0-031737-openshift-default-64b8f46f47-nbdn6\" (UID: \"7088cc53-0ec6-43be-9669-075ca0ad2fa1\") " pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-nbdn6"
Oct 30 13:15:51.046983 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:51.046961    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7088cc53-0ec6-43be-9669-075ca0ad2fa1-istio-token\") pod \"e2e-worker-w0-031737-openshift-default-64b8f46f47-nbdn6\" (UID: \"7088cc53-0ec6-43be-9669-075ca0ad2fa1\") " pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-nbdn6"
Oct 30 13:15:51.047235 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:51.047218    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m2dgf\" (UniqueName: \"kubernetes.io/projected/7088cc53-0ec6-43be-9669-075ca0ad2fa1-kube-api-access-m2dgf\") pod \"e2e-worker-w0-031737-openshift-default-64b8f46f47-nbdn6\" (UID: \"7088cc53-0ec6-43be-9669-075ca0ad2fa1\") " pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-nbdn6"
Oct 30 13:15:51.145603 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:51.145520    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-nbdn6"
Oct 30 13:15:51.329052 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:51.329015    4943 patch_prober.go:29] interesting pod/e2e-worker-w2-f76c09-openshift-default-65d5845b67-kpvbd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused" start-of-body=
Oct 30 13:15:51.329189 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:51.329070    4943 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-kpvbd" podUID="f0e82596-0383-4ec9-a856-a6234e9da808" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused"
Oct 30 13:15:51.506578 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:51.506499    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-nbdn6"]
Oct 30 13:15:51.507908 ip-10-0-142-84 kubenswrapper[4943]: W1030 13:15:51.507880    4943 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7088cc53_0ec6_43be_9669_075ca0ad2fa1.slice/crio-5bd61918f44e0db36444347a19fa0832778195afeeb717408925c3b0f7eb67c9 WatchSource:0}: Error finding container 5bd61918f44e0db36444347a19fa0832778195afeeb717408925c3b0f7eb67c9: Status 404 returned error can't find the container with id 5bd61918f44e0db36444347a19fa0832778195afeeb717408925c3b0f7eb67c9
Oct 30 13:15:51.510702 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:51.510670    4943 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 13:15:51.510750 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:51.510738    4943 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 13:15:51.510788 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:51.510768    4943 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 13:15:51.912311 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:51.912200    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-nbdn6" event={"ID":"7088cc53-0ec6-43be-9669-075ca0ad2fa1","Type":"ContainerStarted","Data":"6ada3baab87084aaeea903d02bf1eaba6e0b57952457b98756be7ec64be7cc28"}
Oct 30 13:15:51.912311 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:51.912292    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-nbdn6" event={"ID":"7088cc53-0ec6-43be-9669-075ca0ad2fa1","Type":"ContainerStarted","Data":"5bd61918f44e0db36444347a19fa0832778195afeeb717408925c3b0f7eb67c9"}
Oct 30 13:15:51.991317 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:51.981543    4943 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-nbdn6" podStartSLOduration=1.981520639 podStartE2EDuration="1.981520639s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:15:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:15:51.96918008 +0000 UTC m=+1127.757183811" watchObservedRunningTime="2026-10-30 13:15:51.981520639 +0000 UTC m=+1127.769524372"
Oct 30 13:15:52.146200 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:52.146162    4943 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-nbdn6"
Oct 30 13:15:52.148204 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:52.148180    4943 patch_prober.go:29] interesting pod/e2e-worker-w0-031737-openshift-default-64b8f46f47-nbdn6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.87:15021/healthz/ready\": dial tcp 10.132.0.87:15021: connect: connection refused" start-of-body=
Oct 30 13:15:52.148347 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:52.148229    4943 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-nbdn6" podUID="7088cc53-0ec6-43be-9669-075ca0ad2fa1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.87:15021/healthz/ready\": dial tcp 10.132.0.87:15021: connect: connection refused"
Oct 30 13:15:52.329207 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:52.329171    4943 patch_prober.go:29] interesting pod/e2e-worker-w2-f76c09-openshift-default-65d5845b67-kpvbd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused" start-of-body=
Oct 30 13:15:52.329417 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:52.329238    4943 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-kpvbd" podUID="f0e82596-0383-4ec9-a856-a6234e9da808" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused"
Oct 30 13:15:53.149764 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:53.149734    4943 patch_prober.go:29] interesting pod/e2e-worker-w0-031737-openshift-default-64b8f46f47-nbdn6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.87:15021/healthz/ready\": dial tcp 10.132.0.87:15021: connect: connection refused" start-of-body=
Oct 30 13:15:53.150402 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:53.150345    4943 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-nbdn6" podUID="7088cc53-0ec6-43be-9669-075ca0ad2fa1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.87:15021/healthz/ready\": dial tcp 10.132.0.87:15021: connect: connection refused"
Oct 30 13:15:53.734286 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:53.732462    4943 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-301cef69-8lmsm"]
Oct 30 13:15:53.758698 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:53.758668    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-301cef69-8lmsm"]
Oct 30 13:15:53.783594 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:53.759764    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-301cef69-8lmsm"
Oct 30 13:15:53.817629 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:53.817597    4943 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-p9z57\""
Oct 30 13:15:53.835442 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:53.835408    4943 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 13:15:53.884521 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:53.865682    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-69jqz\" (UniqueName: \"kubernetes.io/projected/ea5a5d3f-1175-4fe1-bc37-acf4dcc76a62-kube-api-access-69jqz\") pod \"maas-api-revoke-sub-301cef69-8lmsm\" (UID: \"ea5a5d3f-1175-4fe1-bc37-acf4dcc76a62\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-301cef69-8lmsm"
Oct 30 13:15:53.884521 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:53.865783    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/ea5a5d3f-1175-4fe1-bc37-acf4dcc76a62-maas-api-xks-ca\") pod \"maas-api-revoke-sub-301cef69-8lmsm\" (UID: \"ea5a5d3f-1175-4fe1-bc37-acf4dcc76a62\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-301cef69-8lmsm"
Oct 30 13:15:53.884521 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:53.865978    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/ea5a5d3f-1175-4fe1-bc37-acf4dcc76a62-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-301cef69-8lmsm\" (UID: \"ea5a5d3f-1175-4fe1-bc37-acf4dcc76a62\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-301cef69-8lmsm"
Oct 30 13:15:54.022357 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:53.973159    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/ea5a5d3f-1175-4fe1-bc37-acf4dcc76a62-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-301cef69-8lmsm\" (UID: \"ea5a5d3f-1175-4fe1-bc37-acf4dcc76a62\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-301cef69-8lmsm"
Oct 30 13:15:54.022357 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:53.973281    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-69jqz\" (UniqueName: \"kubernetes.io/projected/ea5a5d3f-1175-4fe1-bc37-acf4dcc76a62-kube-api-access-69jqz\") pod \"maas-api-revoke-sub-301cef69-8lmsm\" (UID: \"ea5a5d3f-1175-4fe1-bc37-acf4dcc76a62\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-301cef69-8lmsm"
Oct 30 13:15:54.022357 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:53.973371    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/ea5a5d3f-1175-4fe1-bc37-acf4dcc76a62-maas-api-xks-ca\") pod \"maas-api-revoke-sub-301cef69-8lmsm\" (UID: \"ea5a5d3f-1175-4fe1-bc37-acf4dcc76a62\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-301cef69-8lmsm"
Oct 30 13:15:54.022357 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:53.991156    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/ea5a5d3f-1175-4fe1-bc37-acf4dcc76a62-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-301cef69-8lmsm\" (UID: \"ea5a5d3f-1175-4fe1-bc37-acf4dcc76a62\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-301cef69-8lmsm"
Oct 30 13:15:54.091838 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:54.091798    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/ea5a5d3f-1175-4fe1-bc37-acf4dcc76a62-maas-api-xks-ca\") pod \"maas-api-revoke-sub-301cef69-8lmsm\" (UID: \"ea5a5d3f-1175-4fe1-bc37-acf4dcc76a62\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-301cef69-8lmsm"
Oct 30 13:15:54.143406 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:54.140034    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-69jqz\" (UniqueName: \"kubernetes.io/projected/ea5a5d3f-1175-4fe1-bc37-acf4dcc76a62-kube-api-access-69jqz\") pod \"maas-api-revoke-sub-301cef69-8lmsm\" (UID: \"ea5a5d3f-1175-4fe1-bc37-acf4dcc76a62\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-301cef69-8lmsm"
Oct 30 13:15:54.167435 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:54.167394    4943 patch_prober.go:29] interesting pod/e2e-worker-w0-031737-openshift-default-64b8f46f47-nbdn6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.87:15021/healthz/ready\": dial tcp 10.132.0.87:15021: connect: connection refused" start-of-body=
Oct 30 13:15:54.168059 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:54.168021    4943 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-nbdn6" podUID="7088cc53-0ec6-43be-9669-075ca0ad2fa1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.87:15021/healthz/ready\": dial tcp 10.132.0.87:15021: connect: connection refused"
Oct 30 13:15:54.304337 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:54.303571    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-301cef69-8lmsm"
Oct 30 13:15:54.466669 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:54.466630    4943 patch_prober.go:29] interesting pod/e2e-worker-w2-f76c09-openshift-default-65d5845b67-kpvbd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.85:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 13:15:54.466980 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:54.466945    4943 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-kpvbd" podUID="f0e82596-0383-4ec9-a856-a6234e9da808" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.85:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:15:55.203300 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:55.203236    4943 patch_prober.go:29] interesting pod/e2e-worker-w0-031737-openshift-default-64b8f46f47-nbdn6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.87:15021/healthz/ready\": dial tcp 10.132.0.87:15021: connect: connection refused" start-of-body=
Oct 30 13:15:55.205395 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:55.203834    4943 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-nbdn6" podUID="7088cc53-0ec6-43be-9669-075ca0ad2fa1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.87:15021/healthz/ready\": dial tcp 10.132.0.87:15021: connect: connection refused"
Oct 30 13:15:55.431294 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:55.431231    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-301cef69-8lmsm"]
Oct 30 13:15:55.436384 ip-10-0-142-84 kubenswrapper[4943]: W1030 13:15:55.436334    4943 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podea5a5d3f_1175_4fe1_bc37_acf4dcc76a62.slice/crio-a8825b3d05dbe227da710253d3147ec50f7675f3fae97285d54b9ce749788451 WatchSource:0}: Error finding container a8825b3d05dbe227da710253d3147ec50f7675f3fae97285d54b9ce749788451: Status 404 returned error can't find the container with id a8825b3d05dbe227da710253d3147ec50f7675f3fae97285d54b9ce749788451
Oct 30 13:15:55.470810 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:55.470754    4943 patch_prober.go:29] interesting pod/e2e-worker-w2-f76c09-openshift-default-65d5845b67-kpvbd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.85:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 13:15:55.485627 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:55.485592    4943 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-kpvbd" podUID="f0e82596-0383-4ec9-a856-a6234e9da808" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.85:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:15:55.574590 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:55.574557    4943 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-kpvbd"
Oct 30 13:15:55.576282 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:55.576236    4943 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-kpvbd"
Oct 30 13:15:55.582665 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:55.582629    4943 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-kpvbd"
Oct 30 13:15:56.146227 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:56.146141    4943 patch_prober.go:29] interesting pod/e2e-worker-w0-031737-openshift-default-64b8f46f47-nbdn6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.87:15021/healthz/ready\": dial tcp 10.132.0.87:15021: connect: connection refused" start-of-body=
Oct 30 13:15:56.146227 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:56.146204    4943 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-nbdn6" podUID="7088cc53-0ec6-43be-9669-075ca0ad2fa1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.87:15021/healthz/ready\": dial tcp 10.132.0.87:15021: connect: connection refused"
Oct 30 13:15:56.338372 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:56.338339    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-301cef69-8lmsm" event={"ID":"ea5a5d3f-1175-4fe1-bc37-acf4dcc76a62","Type":"ContainerStarted","Data":"e9709984d4dc46fbb9bea79b5a24418bbb2a51b79d5a15850579e8994bf4c126"}
Oct 30 13:15:56.338877 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:56.338430    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-301cef69-8lmsm" event={"ID":"ea5a5d3f-1175-4fe1-bc37-acf4dcc76a62","Type":"ContainerStarted","Data":"a8825b3d05dbe227da710253d3147ec50f7675f3fae97285d54b9ce749788451"}
Oct 30 13:15:56.403808 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:56.403676    4943 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-301cef69-8lmsm" podStartSLOduration=3.4036555010000002 podStartE2EDuration="3.403655501s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:15:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:15:56.399012062 +0000 UTC m=+1132.187015793" watchObservedRunningTime="2026-10-30 13:15:56.403655501 +0000 UTC m=+1132.191659232"
Oct 30 13:15:57.146228 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:57.146146    4943 patch_prober.go:29] interesting pod/e2e-worker-w0-031737-openshift-default-64b8f46f47-nbdn6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.87:15021/healthz/ready\": dial tcp 10.132.0.87:15021: connect: connection refused" start-of-body=
Oct 30 13:15:57.146399 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:57.146331    4943 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-nbdn6" podUID="7088cc53-0ec6-43be-9669-075ca0ad2fa1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.87:15021/healthz/ready\": dial tcp 10.132.0.87:15021: connect: connection refused"
Oct 30 13:15:57.353417 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:57.353379    4943 generic.go:356] "Generic (PLEG): container finished" podID="ea5a5d3f-1175-4fe1-bc37-acf4dcc76a62" containerID="e9709984d4dc46fbb9bea79b5a24418bbb2a51b79d5a15850579e8994bf4c126" exitCode=0
Oct 30 13:15:57.353805 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:57.353462    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-301cef69-8lmsm" event={"ID":"ea5a5d3f-1175-4fe1-bc37-acf4dcc76a62","Type":"ContainerDied","Data":"e9709984d4dc46fbb9bea79b5a24418bbb2a51b79d5a15850579e8994bf4c126"}
Oct 30 13:15:58.146181 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:58.146146    4943 patch_prober.go:29] interesting pod/e2e-worker-w0-031737-openshift-default-64b8f46f47-nbdn6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.87:15021/healthz/ready\": dial tcp 10.132.0.87:15021: connect: connection refused" start-of-body=
Oct 30 13:15:58.146361 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:58.146212    4943 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-nbdn6" podUID="7088cc53-0ec6-43be-9669-075ca0ad2fa1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.87:15021/healthz/ready\": dial tcp 10.132.0.87:15021: connect: connection refused"
Oct 30 13:15:58.561885 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:58.561857    4943 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-301cef69-8lmsm"
Oct 30 13:15:58.661897 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:58.661791    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/ea5a5d3f-1175-4fe1-bc37-acf4dcc76a62-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/ea5a5d3f-1175-4fe1-bc37-acf4dcc76a62-maas-api-xks-ca\") pod \"ea5a5d3f-1175-4fe1-bc37-acf4dcc76a62\" (UID: \"ea5a5d3f-1175-4fe1-bc37-acf4dcc76a62\") "
Oct 30 13:15:58.662081 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:58.661975    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ea5a5d3f-1175-4fe1-bc37-acf4dcc76a62-kube-api-access-69jqz\" (UniqueName: \"kubernetes.io/projected/ea5a5d3f-1175-4fe1-bc37-acf4dcc76a62-kube-api-access-69jqz\") pod \"ea5a5d3f-1175-4fe1-bc37-acf4dcc76a62\" (UID: \"ea5a5d3f-1175-4fe1-bc37-acf4dcc76a62\") "
Oct 30 13:15:58.662150 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:58.662098    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ea5a5d3f-1175-4fe1-bc37-acf4dcc76a62-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/ea5a5d3f-1175-4fe1-bc37-acf4dcc76a62-maas-api-openshift-ca\") pod \"ea5a5d3f-1175-4fe1-bc37-acf4dcc76a62\" (UID: \"ea5a5d3f-1175-4fe1-bc37-acf4dcc76a62\") "
Oct 30 13:15:58.662476 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:58.662449    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ea5a5d3f-1175-4fe1-bc37-acf4dcc76a62-maas-api-openshift-ca" pod "ea5a5d3f-1175-4fe1-bc37-acf4dcc76a62" (UID: "ea5a5d3f-1175-4fe1-bc37-acf4dcc76a62"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 13:15:58.662674 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:58.662652    4943 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/ea5a5d3f-1175-4fe1-bc37-acf4dcc76a62-maas-api-openshift-ca\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:15:58.664053 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:58.664016    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ea5a5d3f-1175-4fe1-bc37-acf4dcc76a62-maas-api-xks-ca" pod "ea5a5d3f-1175-4fe1-bc37-acf4dcc76a62" (UID: "ea5a5d3f-1175-4fe1-bc37-acf4dcc76a62"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 13:15:58.664631 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:58.664606    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ea5a5d3f-1175-4fe1-bc37-acf4dcc76a62-kube-api-access-69jqz" pod "ea5a5d3f-1175-4fe1-bc37-acf4dcc76a62" (UID: "ea5a5d3f-1175-4fe1-bc37-acf4dcc76a62"). InnerVolumeSpecName "kube-api-access-69jqz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:15:58.764175 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:58.764136    4943 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/ea5a5d3f-1175-4fe1-bc37-acf4dcc76a62-maas-api-xks-ca\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:15:58.764175 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:58.764177    4943 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-69jqz\" (UniqueName: \"kubernetes.io/projected/ea5a5d3f-1175-4fe1-bc37-acf4dcc76a62-kube-api-access-69jqz\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:15:59.150137 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:59.150102    4943 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-nbdn6"
Oct 30 13:15:59.150504 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:59.150487    4943 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-nbdn6"
Oct 30 13:15:59.151100 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:59.151083    4943 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-nbdn6"
Oct 30 13:15:59.411376 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:59.411240    4943 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-301cef69-8lmsm"
Oct 30 13:15:59.411376 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:59.411292    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-301cef69-8lmsm" event={"ID":"ea5a5d3f-1175-4fe1-bc37-acf4dcc76a62","Type":"ContainerDied","Data":"a8825b3d05dbe227da710253d3147ec50f7675f3fae97285d54b9ce749788451"}
Oct 30 13:15:59.411376 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:59.411357    4943 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a8825b3d05dbe227da710253d3147ec50f7675f3fae97285d54b9ce749788451"
Oct 30 13:15:59.414020 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:59.413996    4943 generic.go:356] "Generic (PLEG): container finished" podID="1494fd10-5e14-4d43-aeaf-08e3d779b232" containerID="244536df8fce8dcc781a0e7c315b46d6bbed2fdc13ce9e3072bcaf2e5df688eb" exitCode=0
Oct 30 13:15:59.414353 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:15:59.414325    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-lfhqw-rcd8z" event={"ID":"1494fd10-5e14-4d43-aeaf-08e3d779b232","Type":"ContainerDied","Data":"244536df8fce8dcc781a0e7c315b46d6bbed2fdc13ce9e3072bcaf2e5df688eb"}
Oct 30 13:16:00.587976 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:00.587927    4943 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-lfhqw-rcd8z"
Oct 30 13:16:00.685193 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:00.685157    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1494fd10-5e14-4d43-aeaf-08e3d779b232-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1494fd10-5e14-4d43-aeaf-08e3d779b232-service-ca-bundle\") pod \"1494fd10-5e14-4d43-aeaf-08e3d779b232\" (UID: \"1494fd10-5e14-4d43-aeaf-08e3d779b232\") "
Oct 30 13:16:00.685364 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:00.685339    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1494fd10-5e14-4d43-aeaf-08e3d779b232-archives-path\" (UniqueName: \"kubernetes.io/empty-dir/1494fd10-5e14-4d43-aeaf-08e3d779b232-archives-path\") pod \"1494fd10-5e14-4d43-aeaf-08e3d779b232\" (UID: \"1494fd10-5e14-4d43-aeaf-08e3d779b232\") "
Oct 30 13:16:00.685424 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:00.685412    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1494fd10-5e14-4d43-aeaf-08e3d779b232-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1494fd10-5e14-4d43-aeaf-08e3d779b232-trusted-ca-bundle\") pod \"1494fd10-5e14-4d43-aeaf-08e3d779b232\" (UID: \"1494fd10-5e14-4d43-aeaf-08e3d779b232\") "
Oct 30 13:16:00.685495 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:00.685479    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1494fd10-5e14-4d43-aeaf-08e3d779b232-kube-api-access-jkdm6\" (UniqueName: \"kubernetes.io/projected/1494fd10-5e14-4d43-aeaf-08e3d779b232-kube-api-access-jkdm6\") pod \"1494fd10-5e14-4d43-aeaf-08e3d779b232\" (UID: \"1494fd10-5e14-4d43-aeaf-08e3d779b232\") "
Oct 30 13:16:00.685720 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:00.685690    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1494fd10-5e14-4d43-aeaf-08e3d779b232-archives-path" pod "1494fd10-5e14-4d43-aeaf-08e3d779b232" (UID: "1494fd10-5e14-4d43-aeaf-08e3d779b232"). InnerVolumeSpecName "archives-path". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:16:00.685803 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:00.685724    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1494fd10-5e14-4d43-aeaf-08e3d779b232-service-ca-bundle" pod "1494fd10-5e14-4d43-aeaf-08e3d779b232" (UID: "1494fd10-5e14-4d43-aeaf-08e3d779b232"). InnerVolumeSpecName "service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 13:16:00.685803 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:00.685782    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1494fd10-5e14-4d43-aeaf-08e3d779b232-trusted-ca-bundle" pod "1494fd10-5e14-4d43-aeaf-08e3d779b232" (UID: "1494fd10-5e14-4d43-aeaf-08e3d779b232"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 13:16:00.685972 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:00.685952    4943 reconciler_common.go:299] "Volume detached for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1494fd10-5e14-4d43-aeaf-08e3d779b232-service-ca-bundle\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:16:00.686012 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:00.685980    4943 reconciler_common.go:299] "Volume detached for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/1494fd10-5e14-4d43-aeaf-08e3d779b232-archives-path\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:16:00.686012 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:00.685995    4943 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1494fd10-5e14-4d43-aeaf-08e3d779b232-trusted-ca-bundle\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:16:00.687668 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:00.687643    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1494fd10-5e14-4d43-aeaf-08e3d779b232-kube-api-access-jkdm6" pod "1494fd10-5e14-4d43-aeaf-08e3d779b232" (UID: "1494fd10-5e14-4d43-aeaf-08e3d779b232"). InnerVolumeSpecName "kube-api-access-jkdm6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:16:00.787476 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:00.787445    4943 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jkdm6\" (UniqueName: \"kubernetes.io/projected/1494fd10-5e14-4d43-aeaf-08e3d779b232-kube-api-access-jkdm6\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:16:00.868880 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:00.868799    4943 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w3-d0a18c/distinct-w3-d0a18c-kserve-86479f6bbf-5zcqq"]
Oct 30 13:16:00.869466 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:00.869444    4943 cpu_manager.go:414] "containerMap: removing container" podUID="1494fd10-5e14-4d43-aeaf-08e3d779b232" containerName="insights-gathering"
Oct 30 13:16:00.869584 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:00.869466    4943 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1494fd10-5e14-4d43-aeaf-08e3d779b232" containerName="insights-gathering"
Oct 30 13:16:00.869584 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:00.869514    4943 cpu_manager.go:414] "containerMap: removing container" podUID="ea5a5d3f-1175-4fe1-bc37-acf4dcc76a62" containerName="revoke-keys"
Oct 30 13:16:00.869584 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:00.869522    4943 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ea5a5d3f-1175-4fe1-bc37-acf4dcc76a62" containerName="revoke-keys"
Oct 30 13:16:00.869744 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:00.869677    4943 memory_manager.go:365] "RemoveStaleState removing state" podUID="1494fd10-5e14-4d43-aeaf-08e3d779b232" containerName="insights-gathering"
Oct 30 13:16:00.869744 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:00.869696    4943 memory_manager.go:365] "RemoveStaleState removing state" podUID="ea5a5d3f-1175-4fe1-bc37-acf4dcc76a62" containerName="revoke-keys"
Oct 30 13:16:00.873450 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:00.873427    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-d0a18c/distinct-w3-d0a18c-kserve-86479f6bbf-5zcqq"
Oct 30 13:16:00.890564 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:00.890520    4943 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w3-d0a18c\"/\"kube-root-ca.crt\""
Oct 30 13:16:00.890685 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:00.890650    4943 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w3-d0a18c\"/\"default-dockercfg-hwmgv\""
Oct 30 13:16:00.890837 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:00.890802    4943 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w3-d0a18c\"/\"openshift-service-ca.crt\""
Oct 30 13:16:00.894847 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:00.894820    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w3-d0a18c/distinct-w3-d0a18c-kserve-86479f6bbf-5zcqq"]
Oct 30 13:16:00.904204 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:00.904185    4943 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w3-d0a18c\"/\"distinct-w3-d0a18c-kserve-self-signed-certs\""
Oct 30 13:16:00.990446 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:00.990417    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/63dace4c-7832-47dd-84ab-2c91483a5563-dshm\") pod \"distinct-w3-d0a18c-kserve-86479f6bbf-5zcqq\" (UID: \"63dace4c-7832-47dd-84ab-2c91483a5563\") " pod="e2e-models-e2e-worker-w3-d0a18c/distinct-w3-d0a18c-kserve-86479f6bbf-5zcqq"
Oct 30 13:16:00.990584 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:00.990450    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/63dace4c-7832-47dd-84ab-2c91483a5563-model-cache\") pod \"distinct-w3-d0a18c-kserve-86479f6bbf-5zcqq\" (UID: \"63dace4c-7832-47dd-84ab-2c91483a5563\") " pod="e2e-models-e2e-worker-w3-d0a18c/distinct-w3-d0a18c-kserve-86479f6bbf-5zcqq"
Oct 30 13:16:00.990584 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:00.990556    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/63dace4c-7832-47dd-84ab-2c91483a5563-home\") pod \"distinct-w3-d0a18c-kserve-86479f6bbf-5zcqq\" (UID: \"63dace4c-7832-47dd-84ab-2c91483a5563\") " pod="e2e-models-e2e-worker-w3-d0a18c/distinct-w3-d0a18c-kserve-86479f6bbf-5zcqq"
Oct 30 13:16:00.990682 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:00.990605    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/63dace4c-7832-47dd-84ab-2c91483a5563-tmp-dir\") pod \"distinct-w3-d0a18c-kserve-86479f6bbf-5zcqq\" (UID: \"63dace4c-7832-47dd-84ab-2c91483a5563\") " pod="e2e-models-e2e-worker-w3-d0a18c/distinct-w3-d0a18c-kserve-86479f6bbf-5zcqq"
Oct 30 13:16:00.990682 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:00.990639    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/63dace4c-7832-47dd-84ab-2c91483a5563-tls-certs\") pod \"distinct-w3-d0a18c-kserve-86479f6bbf-5zcqq\" (UID: \"63dace4c-7832-47dd-84ab-2c91483a5563\") " pod="e2e-models-e2e-worker-w3-d0a18c/distinct-w3-d0a18c-kserve-86479f6bbf-5zcqq"
Oct 30 13:16:00.990807 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:00.990782    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d8lw2\" (UniqueName: \"kubernetes.io/projected/63dace4c-7832-47dd-84ab-2c91483a5563-kube-api-access-d8lw2\") pod \"distinct-w3-d0a18c-kserve-86479f6bbf-5zcqq\" (UID: \"63dace4c-7832-47dd-84ab-2c91483a5563\") " pod="e2e-models-e2e-worker-w3-d0a18c/distinct-w3-d0a18c-kserve-86479f6bbf-5zcqq"
Oct 30 13:16:01.094984 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:01.094936    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/63dace4c-7832-47dd-84ab-2c91483a5563-home\") pod \"distinct-w3-d0a18c-kserve-86479f6bbf-5zcqq\" (UID: \"63dace4c-7832-47dd-84ab-2c91483a5563\") " pod="e2e-models-e2e-worker-w3-d0a18c/distinct-w3-d0a18c-kserve-86479f6bbf-5zcqq"
Oct 30 13:16:01.095308 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:01.095282    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/63dace4c-7832-47dd-84ab-2c91483a5563-tmp-dir\") pod \"distinct-w3-d0a18c-kserve-86479f6bbf-5zcqq\" (UID: \"63dace4c-7832-47dd-84ab-2c91483a5563\") " pod="e2e-models-e2e-worker-w3-d0a18c/distinct-w3-d0a18c-kserve-86479f6bbf-5zcqq"
Oct 30 13:16:01.095489 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:01.095468    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/63dace4c-7832-47dd-84ab-2c91483a5563-tls-certs\") pod \"distinct-w3-d0a18c-kserve-86479f6bbf-5zcqq\" (UID: \"63dace4c-7832-47dd-84ab-2c91483a5563\") " pod="e2e-models-e2e-worker-w3-d0a18c/distinct-w3-d0a18c-kserve-86479f6bbf-5zcqq"
Oct 30 13:16:01.095690 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:01.095673    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d8lw2\" (UniqueName: \"kubernetes.io/projected/63dace4c-7832-47dd-84ab-2c91483a5563-kube-api-access-d8lw2\") pod \"distinct-w3-d0a18c-kserve-86479f6bbf-5zcqq\" (UID: \"63dace4c-7832-47dd-84ab-2c91483a5563\") " pod="e2e-models-e2e-worker-w3-d0a18c/distinct-w3-d0a18c-kserve-86479f6bbf-5zcqq"
Oct 30 13:16:01.095877 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:01.095862    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/63dace4c-7832-47dd-84ab-2c91483a5563-dshm\") pod \"distinct-w3-d0a18c-kserve-86479f6bbf-5zcqq\" (UID: \"63dace4c-7832-47dd-84ab-2c91483a5563\") " pod="e2e-models-e2e-worker-w3-d0a18c/distinct-w3-d0a18c-kserve-86479f6bbf-5zcqq"
Oct 30 13:16:01.095995 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:01.095977    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/63dace4c-7832-47dd-84ab-2c91483a5563-model-cache\") pod \"distinct-w3-d0a18c-kserve-86479f6bbf-5zcqq\" (UID: \"63dace4c-7832-47dd-84ab-2c91483a5563\") " pod="e2e-models-e2e-worker-w3-d0a18c/distinct-w3-d0a18c-kserve-86479f6bbf-5zcqq"
Oct 30 13:16:01.096630 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:01.096608    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/63dace4c-7832-47dd-84ab-2c91483a5563-model-cache\") pod \"distinct-w3-d0a18c-kserve-86479f6bbf-5zcqq\" (UID: \"63dace4c-7832-47dd-84ab-2c91483a5563\") " pod="e2e-models-e2e-worker-w3-d0a18c/distinct-w3-d0a18c-kserve-86479f6bbf-5zcqq"
Oct 30 13:16:01.097021 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:01.097001    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/63dace4c-7832-47dd-84ab-2c91483a5563-home\") pod \"distinct-w3-d0a18c-kserve-86479f6bbf-5zcqq\" (UID: \"63dace4c-7832-47dd-84ab-2c91483a5563\") " pod="e2e-models-e2e-worker-w3-d0a18c/distinct-w3-d0a18c-kserve-86479f6bbf-5zcqq"
Oct 30 13:16:01.097410 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:01.097391    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/63dace4c-7832-47dd-84ab-2c91483a5563-tmp-dir\") pod \"distinct-w3-d0a18c-kserve-86479f6bbf-5zcqq\" (UID: \"63dace4c-7832-47dd-84ab-2c91483a5563\") " pod="e2e-models-e2e-worker-w3-d0a18c/distinct-w3-d0a18c-kserve-86479f6bbf-5zcqq"
Oct 30 13:16:01.114454 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:01.114422    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/63dace4c-7832-47dd-84ab-2c91483a5563-tls-certs\") pod \"distinct-w3-d0a18c-kserve-86479f6bbf-5zcqq\" (UID: \"63dace4c-7832-47dd-84ab-2c91483a5563\") " pod="e2e-models-e2e-worker-w3-d0a18c/distinct-w3-d0a18c-kserve-86479f6bbf-5zcqq"
Oct 30 13:16:01.133431 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:01.133324    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d8lw2\" (UniqueName: \"kubernetes.io/projected/63dace4c-7832-47dd-84ab-2c91483a5563-kube-api-access-d8lw2\") pod \"distinct-w3-d0a18c-kserve-86479f6bbf-5zcqq\" (UID: \"63dace4c-7832-47dd-84ab-2c91483a5563\") " pod="e2e-models-e2e-worker-w3-d0a18c/distinct-w3-d0a18c-kserve-86479f6bbf-5zcqq"
Oct 30 13:16:01.136357 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:01.135607    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/63dace4c-7832-47dd-84ab-2c91483a5563-dshm\") pod \"distinct-w3-d0a18c-kserve-86479f6bbf-5zcqq\" (UID: \"63dace4c-7832-47dd-84ab-2c91483a5563\") " pod="e2e-models-e2e-worker-w3-d0a18c/distinct-w3-d0a18c-kserve-86479f6bbf-5zcqq"
Oct 30 13:16:01.190706 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:01.190667    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-d0a18c/distinct-w3-d0a18c-kserve-86479f6bbf-5zcqq"
Oct 30 13:16:01.429620 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:01.429584    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-lfhqw-rcd8z" event={"ID":"1494fd10-5e14-4d43-aeaf-08e3d779b232","Type":"ContainerDied","Data":"a1166ce031876db5c135a72996c45da9e61c360753e7092a66c63e2ee01c62a9"}
Oct 30 13:16:01.429771 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:01.429621    4943 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-lfhqw-rcd8z"
Oct 30 13:16:01.429771 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:01.429634    4943 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a1166ce031876db5c135a72996c45da9e61c360753e7092a66c63e2ee01c62a9"
Oct 30 13:16:01.517985 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:01.517957    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w3-d0a18c/distinct-w3-d0a18c-kserve-86479f6bbf-5zcqq"]
Oct 30 13:16:01.518101 ip-10-0-142-84 kubenswrapper[4943]: W1030 13:16:01.517979    4943 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod63dace4c_7832_47dd_84ab_2c91483a5563.slice/crio-21b6086af0982832b49a290560002b8b51df555345fb1df9e6a20f727195e6a8 WatchSource:0}: Error finding container 21b6086af0982832b49a290560002b8b51df555345fb1df9e6a20f727195e6a8: Status 404 returned error can't find the container with id 21b6086af0982832b49a290560002b8b51df555345fb1df9e6a20f727195e6a8
Oct 30 13:16:02.438799 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:02.438760    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-d0a18c/distinct-w3-d0a18c-kserve-86479f6bbf-5zcqq" event={"ID":"63dace4c-7832-47dd-84ab-2c91483a5563","Type":"ContainerStarted","Data":"adeb6ab24fe6238092db30053af4cd8cc74a617b87301baf954b8382f346cd18"}
Oct 30 13:16:02.439175 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:02.438807    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-d0a18c/distinct-w3-d0a18c-kserve-86479f6bbf-5zcqq" event={"ID":"63dace4c-7832-47dd-84ab-2c91483a5563","Type":"ContainerStarted","Data":"21b6086af0982832b49a290560002b8b51df555345fb1df9e6a20f727195e6a8"}
Oct 30 13:16:02.487941 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:02.487891    4943 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w3-d0a18c/distinct-w3-d0a18c-kserve-86479f6bbf-5zcqq" podStartSLOduration=2.487878529 podStartE2EDuration="2.487878529s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:16:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:16:02.487187436 +0000 UTC m=+1138.275191167" watchObservedRunningTime="2026-10-30 13:16:02.487878529 +0000 UTC m=+1138.275882259"
Oct 30 13:16:03.191611 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:03.191581    4943 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-d0a18c/distinct-w3-d0a18c-kserve-86479f6bbf-5zcqq"
Oct 30 13:16:03.321226 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:03.321185    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-hdmrm"]
Oct 30 13:16:05.955409 ip-10-0-142-84 kubenswrapper[4943]: E1030 13:16:05.955368    4943 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-knqz6" containerName="extractor"
Oct 30 13:16:11.191333 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:11.191296    4943 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w3-d0a18c/distinct-w3-d0a18c-kserve-86479f6bbf-5zcqq"
Oct 30 13:16:11.206756 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:11.206728    4943 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w3-d0a18c/distinct-w3-d0a18c-kserve-86479f6bbf-5zcqq"
Oct 30 13:16:11.572317 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:11.572234    4943 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-d0a18c/distinct-w3-d0a18c-kserve-86479f6bbf-5zcqq"
Oct 30 13:16:21.572765 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:21.572731    4943 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-glf25"]
Oct 30 13:16:21.577245 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:21.577221    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-glf25"
Oct 30 13:16:21.605553 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:21.605524    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-glf25"]
Oct 30 13:16:21.621417 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:21.621390    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dbc82b1c-1c45-4263-b8ee-090fbecff086-istio-data\") pod \"e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-glf25\" (UID: \"dbc82b1c-1c45-4263-b8ee-090fbecff086\") " pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-glf25"
Oct 30 13:16:21.621531 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:21.621445    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dbc82b1c-1c45-4263-b8ee-090fbecff086-istiod-ca-cert\") pod \"e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-glf25\" (UID: \"dbc82b1c-1c45-4263-b8ee-090fbecff086\") " pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-glf25"
Oct 30 13:16:21.621531 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:21.621506    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dbc82b1c-1c45-4263-b8ee-090fbecff086-workload-socket\") pod \"e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-glf25\" (UID: \"dbc82b1c-1c45-4263-b8ee-090fbecff086\") " pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-glf25"
Oct 30 13:16:21.621610 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:21.621542    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dbc82b1c-1c45-4263-b8ee-090fbecff086-credential-socket\") pod \"e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-glf25\" (UID: \"dbc82b1c-1c45-4263-b8ee-090fbecff086\") " pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-glf25"
Oct 30 13:16:21.621610 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:21.621572    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dbc82b1c-1c45-4263-b8ee-090fbecff086-workload-certs\") pod \"e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-glf25\" (UID: \"dbc82b1c-1c45-4263-b8ee-090fbecff086\") " pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-glf25"
Oct 30 13:16:21.621684 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:21.621668    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dbc82b1c-1c45-4263-b8ee-090fbecff086-istio-podinfo\") pod \"e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-glf25\" (UID: \"dbc82b1c-1c45-4263-b8ee-090fbecff086\") " pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-glf25"
Oct 30 13:16:21.621723 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:21.621690    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f2jml\" (UniqueName: \"kubernetes.io/projected/dbc82b1c-1c45-4263-b8ee-090fbecff086-kube-api-access-f2jml\") pod \"e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-glf25\" (UID: \"dbc82b1c-1c45-4263-b8ee-090fbecff086\") " pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-glf25"
Oct 30 13:16:21.621798 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:21.621775    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dbc82b1c-1c45-4263-b8ee-090fbecff086-istio-envoy\") pod \"e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-glf25\" (UID: \"dbc82b1c-1c45-4263-b8ee-090fbecff086\") " pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-glf25"
Oct 30 13:16:21.621876 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:21.621861    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dbc82b1c-1c45-4263-b8ee-090fbecff086-istio-token\") pod \"e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-glf25\" (UID: \"dbc82b1c-1c45-4263-b8ee-090fbecff086\") " pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-glf25"
Oct 30 13:16:21.722931 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:21.722897    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dbc82b1c-1c45-4263-b8ee-090fbecff086-credential-socket\") pod \"e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-glf25\" (UID: \"dbc82b1c-1c45-4263-b8ee-090fbecff086\") " pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-glf25"
Oct 30 13:16:21.723080 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:21.722950    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dbc82b1c-1c45-4263-b8ee-090fbecff086-workload-certs\") pod \"e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-glf25\" (UID: \"dbc82b1c-1c45-4263-b8ee-090fbecff086\") " pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-glf25"
Oct 30 13:16:21.723080 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:21.723007    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dbc82b1c-1c45-4263-b8ee-090fbecff086-istio-podinfo\") pod \"e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-glf25\" (UID: \"dbc82b1c-1c45-4263-b8ee-090fbecff086\") " pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-glf25"
Oct 30 13:16:21.723080 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:21.723033    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f2jml\" (UniqueName: \"kubernetes.io/projected/dbc82b1c-1c45-4263-b8ee-090fbecff086-kube-api-access-f2jml\") pod \"e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-glf25\" (UID: \"dbc82b1c-1c45-4263-b8ee-090fbecff086\") " pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-glf25"
Oct 30 13:16:21.723226 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:21.723108    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dbc82b1c-1c45-4263-b8ee-090fbecff086-istio-envoy\") pod \"e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-glf25\" (UID: \"dbc82b1c-1c45-4263-b8ee-090fbecff086\") " pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-glf25"
Oct 30 13:16:21.723226 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:21.723141    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dbc82b1c-1c45-4263-b8ee-090fbecff086-istio-token\") pod \"e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-glf25\" (UID: \"dbc82b1c-1c45-4263-b8ee-090fbecff086\") " pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-glf25"
Oct 30 13:16:21.723386 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:21.723225    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dbc82b1c-1c45-4263-b8ee-090fbecff086-istio-data\") pod \"e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-glf25\" (UID: \"dbc82b1c-1c45-4263-b8ee-090fbecff086\") " pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-glf25"
Oct 30 13:16:21.723386 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:21.723329    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dbc82b1c-1c45-4263-b8ee-090fbecff086-istiod-ca-cert\") pod \"e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-glf25\" (UID: \"dbc82b1c-1c45-4263-b8ee-090fbecff086\") " pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-glf25"
Oct 30 13:16:21.723386 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:21.723332    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dbc82b1c-1c45-4263-b8ee-090fbecff086-credential-socket\") pod \"e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-glf25\" (UID: \"dbc82b1c-1c45-4263-b8ee-090fbecff086\") " pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-glf25"
Oct 30 13:16:21.723386 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:21.723359    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dbc82b1c-1c45-4263-b8ee-090fbecff086-workload-socket\") pod \"e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-glf25\" (UID: \"dbc82b1c-1c45-4263-b8ee-090fbecff086\") " pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-glf25"
Oct 30 13:16:21.723606 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:21.723399    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dbc82b1c-1c45-4263-b8ee-090fbecff086-workload-certs\") pod \"e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-glf25\" (UID: \"dbc82b1c-1c45-4263-b8ee-090fbecff086\") " pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-glf25"
Oct 30 13:16:21.723659 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:21.723603    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dbc82b1c-1c45-4263-b8ee-090fbecff086-istio-data\") pod \"e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-glf25\" (UID: \"dbc82b1c-1c45-4263-b8ee-090fbecff086\") " pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-glf25"
Oct 30 13:16:21.723659 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:21.723650    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dbc82b1c-1c45-4263-b8ee-090fbecff086-workload-socket\") pod \"e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-glf25\" (UID: \"dbc82b1c-1c45-4263-b8ee-090fbecff086\") " pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-glf25"
Oct 30 13:16:21.723971 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:21.723949    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dbc82b1c-1c45-4263-b8ee-090fbecff086-istiod-ca-cert\") pod \"e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-glf25\" (UID: \"dbc82b1c-1c45-4263-b8ee-090fbecff086\") " pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-glf25"
Oct 30 13:16:21.725369 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:21.725338    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dbc82b1c-1c45-4263-b8ee-090fbecff086-istio-envoy\") pod \"e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-glf25\" (UID: \"dbc82b1c-1c45-4263-b8ee-090fbecff086\") " pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-glf25"
Oct 30 13:16:21.725689 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:21.725665    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dbc82b1c-1c45-4263-b8ee-090fbecff086-istio-podinfo\") pod \"e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-glf25\" (UID: \"dbc82b1c-1c45-4263-b8ee-090fbecff086\") " pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-glf25"
Oct 30 13:16:21.748751 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:21.748729    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dbc82b1c-1c45-4263-b8ee-090fbecff086-istio-token\") pod \"e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-glf25\" (UID: \"dbc82b1c-1c45-4263-b8ee-090fbecff086\") " pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-glf25"
Oct 30 13:16:21.748853 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:21.748782    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f2jml\" (UniqueName: \"kubernetes.io/projected/dbc82b1c-1c45-4263-b8ee-090fbecff086-kube-api-access-f2jml\") pod \"e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-glf25\" (UID: \"dbc82b1c-1c45-4263-b8ee-090fbecff086\") " pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-glf25"
Oct 30 13:16:21.806661 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:21.806630    4943 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w1-05942d/distinct-w1-05942d-kserve-5fb44dc664-mcbkc"]
Oct 30 13:16:21.812589 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:21.812560    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-05942d/distinct-w1-05942d-kserve-5fb44dc664-mcbkc"
Oct 30 13:16:21.832695 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:21.832634    4943 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w1-05942d\"/\"kube-root-ca.crt\""
Oct 30 13:16:21.834433 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:21.834406    4943 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w1-05942d\"/\"default-dockercfg-vp8gb\""
Oct 30 13:16:21.834548 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:21.834443    4943 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w1-05942d\"/\"distinct-w1-05942d-kserve-self-signed-certs\""
Oct 30 13:16:21.834604 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:21.834545    4943 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w1-05942d\"/\"openshift-service-ca.crt\""
Oct 30 13:16:21.837499 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:21.837476    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w1-05942d/distinct-w1-05942d-kserve-5fb44dc664-mcbkc"]
Oct 30 13:16:21.893955 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:21.893923    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-glf25"
Oct 30 13:16:21.925451 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:21.925417    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xlflg\" (UniqueName: \"kubernetes.io/projected/d08eb359-cf9e-422c-a3a7-35fd173ab7b6-kube-api-access-xlflg\") pod \"distinct-w1-05942d-kserve-5fb44dc664-mcbkc\" (UID: \"d08eb359-cf9e-422c-a3a7-35fd173ab7b6\") " pod="e2e-models-e2e-worker-w1-05942d/distinct-w1-05942d-kserve-5fb44dc664-mcbkc"
Oct 30 13:16:21.925604 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:21.925471    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d08eb359-cf9e-422c-a3a7-35fd173ab7b6-tls-certs\") pod \"distinct-w1-05942d-kserve-5fb44dc664-mcbkc\" (UID: \"d08eb359-cf9e-422c-a3a7-35fd173ab7b6\") " pod="e2e-models-e2e-worker-w1-05942d/distinct-w1-05942d-kserve-5fb44dc664-mcbkc"
Oct 30 13:16:21.925604 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:21.925547    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d08eb359-cf9e-422c-a3a7-35fd173ab7b6-dshm\") pod \"distinct-w1-05942d-kserve-5fb44dc664-mcbkc\" (UID: \"d08eb359-cf9e-422c-a3a7-35fd173ab7b6\") " pod="e2e-models-e2e-worker-w1-05942d/distinct-w1-05942d-kserve-5fb44dc664-mcbkc"
Oct 30 13:16:21.925604 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:21.925571    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d08eb359-cf9e-422c-a3a7-35fd173ab7b6-tmp-dir\") pod \"distinct-w1-05942d-kserve-5fb44dc664-mcbkc\" (UID: \"d08eb359-cf9e-422c-a3a7-35fd173ab7b6\") " pod="e2e-models-e2e-worker-w1-05942d/distinct-w1-05942d-kserve-5fb44dc664-mcbkc"
Oct 30 13:16:21.925779 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:21.925615    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d08eb359-cf9e-422c-a3a7-35fd173ab7b6-model-cache\") pod \"distinct-w1-05942d-kserve-5fb44dc664-mcbkc\" (UID: \"d08eb359-cf9e-422c-a3a7-35fd173ab7b6\") " pod="e2e-models-e2e-worker-w1-05942d/distinct-w1-05942d-kserve-5fb44dc664-mcbkc"
Oct 30 13:16:21.925836 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:21.925792    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d08eb359-cf9e-422c-a3a7-35fd173ab7b6-home\") pod \"distinct-w1-05942d-kserve-5fb44dc664-mcbkc\" (UID: \"d08eb359-cf9e-422c-a3a7-35fd173ab7b6\") " pod="e2e-models-e2e-worker-w1-05942d/distinct-w1-05942d-kserve-5fb44dc664-mcbkc"
Oct 30 13:16:22.038291 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:22.035771    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d08eb359-cf9e-422c-a3a7-35fd173ab7b6-model-cache\") pod \"distinct-w1-05942d-kserve-5fb44dc664-mcbkc\" (UID: \"d08eb359-cf9e-422c-a3a7-35fd173ab7b6\") " pod="e2e-models-e2e-worker-w1-05942d/distinct-w1-05942d-kserve-5fb44dc664-mcbkc"
Oct 30 13:16:22.038291 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:22.035831    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d08eb359-cf9e-422c-a3a7-35fd173ab7b6-home\") pod \"distinct-w1-05942d-kserve-5fb44dc664-mcbkc\" (UID: \"d08eb359-cf9e-422c-a3a7-35fd173ab7b6\") " pod="e2e-models-e2e-worker-w1-05942d/distinct-w1-05942d-kserve-5fb44dc664-mcbkc"
Oct 30 13:16:22.038291 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:22.036055    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xlflg\" (UniqueName: \"kubernetes.io/projected/d08eb359-cf9e-422c-a3a7-35fd173ab7b6-kube-api-access-xlflg\") pod \"distinct-w1-05942d-kserve-5fb44dc664-mcbkc\" (UID: \"d08eb359-cf9e-422c-a3a7-35fd173ab7b6\") " pod="e2e-models-e2e-worker-w1-05942d/distinct-w1-05942d-kserve-5fb44dc664-mcbkc"
Oct 30 13:16:22.038291 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:22.036109    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d08eb359-cf9e-422c-a3a7-35fd173ab7b6-tls-certs\") pod \"distinct-w1-05942d-kserve-5fb44dc664-mcbkc\" (UID: \"d08eb359-cf9e-422c-a3a7-35fd173ab7b6\") " pod="e2e-models-e2e-worker-w1-05942d/distinct-w1-05942d-kserve-5fb44dc664-mcbkc"
Oct 30 13:16:22.038291 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:22.036134    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d08eb359-cf9e-422c-a3a7-35fd173ab7b6-dshm\") pod \"distinct-w1-05942d-kserve-5fb44dc664-mcbkc\" (UID: \"d08eb359-cf9e-422c-a3a7-35fd173ab7b6\") " pod="e2e-models-e2e-worker-w1-05942d/distinct-w1-05942d-kserve-5fb44dc664-mcbkc"
Oct 30 13:16:22.038291 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:22.036165    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d08eb359-cf9e-422c-a3a7-35fd173ab7b6-tmp-dir\") pod \"distinct-w1-05942d-kserve-5fb44dc664-mcbkc\" (UID: \"d08eb359-cf9e-422c-a3a7-35fd173ab7b6\") " pod="e2e-models-e2e-worker-w1-05942d/distinct-w1-05942d-kserve-5fb44dc664-mcbkc"
Oct 30 13:16:22.038291 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:22.036728    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d08eb359-cf9e-422c-a3a7-35fd173ab7b6-tmp-dir\") pod \"distinct-w1-05942d-kserve-5fb44dc664-mcbkc\" (UID: \"d08eb359-cf9e-422c-a3a7-35fd173ab7b6\") " pod="e2e-models-e2e-worker-w1-05942d/distinct-w1-05942d-kserve-5fb44dc664-mcbkc"
Oct 30 13:16:22.038291 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:22.037003    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d08eb359-cf9e-422c-a3a7-35fd173ab7b6-model-cache\") pod \"distinct-w1-05942d-kserve-5fb44dc664-mcbkc\" (UID: \"d08eb359-cf9e-422c-a3a7-35fd173ab7b6\") " pod="e2e-models-e2e-worker-w1-05942d/distinct-w1-05942d-kserve-5fb44dc664-mcbkc"
Oct 30 13:16:22.056360 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:22.055747    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d08eb359-cf9e-422c-a3a7-35fd173ab7b6-tls-certs\") pod \"distinct-w1-05942d-kserve-5fb44dc664-mcbkc\" (UID: \"d08eb359-cf9e-422c-a3a7-35fd173ab7b6\") " pod="e2e-models-e2e-worker-w1-05942d/distinct-w1-05942d-kserve-5fb44dc664-mcbkc"
Oct 30 13:16:22.061163 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:22.059102    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d08eb359-cf9e-422c-a3a7-35fd173ab7b6-dshm\") pod \"distinct-w1-05942d-kserve-5fb44dc664-mcbkc\" (UID: \"d08eb359-cf9e-422c-a3a7-35fd173ab7b6\") " pod="e2e-models-e2e-worker-w1-05942d/distinct-w1-05942d-kserve-5fb44dc664-mcbkc"
Oct 30 13:16:22.080486 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:22.080398    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d08eb359-cf9e-422c-a3a7-35fd173ab7b6-home\") pod \"distinct-w1-05942d-kserve-5fb44dc664-mcbkc\" (UID: \"d08eb359-cf9e-422c-a3a7-35fd173ab7b6\") " pod="e2e-models-e2e-worker-w1-05942d/distinct-w1-05942d-kserve-5fb44dc664-mcbkc"
Oct 30 13:16:22.086511 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:22.086379    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xlflg\" (UniqueName: \"kubernetes.io/projected/d08eb359-cf9e-422c-a3a7-35fd173ab7b6-kube-api-access-xlflg\") pod \"distinct-w1-05942d-kserve-5fb44dc664-mcbkc\" (UID: \"d08eb359-cf9e-422c-a3a7-35fd173ab7b6\") " pod="e2e-models-e2e-worker-w1-05942d/distinct-w1-05942d-kserve-5fb44dc664-mcbkc"
Oct 30 13:16:22.127922 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:22.127697    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-05942d/distinct-w1-05942d-kserve-5fb44dc664-mcbkc"
Oct 30 13:16:22.225228 ip-10-0-142-84 kubenswrapper[4943]: W1030 13:16:22.225188    4943 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddbc82b1c_1c45_4263_b8ee_090fbecff086.slice/crio-e5ffcccf9ca8b7194cd890fe37f69da356a7be73f7ac72f5e477b8608f953777 WatchSource:0}: Error finding container e5ffcccf9ca8b7194cd890fe37f69da356a7be73f7ac72f5e477b8608f953777: Status 404 returned error can't find the container with id e5ffcccf9ca8b7194cd890fe37f69da356a7be73f7ac72f5e477b8608f953777
Oct 30 13:16:22.226894 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:22.226863    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-glf25"]
Oct 30 13:16:22.228127 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:22.228095    4943 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 13:16:22.228213 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:22.228160    4943 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 13:16:22.228213 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:22.228194    4943 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 13:16:22.440941 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:22.440912    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w1-05942d/distinct-w1-05942d-kserve-5fb44dc664-mcbkc"]
Oct 30 13:16:22.442595 ip-10-0-142-84 kubenswrapper[4943]: W1030 13:16:22.442568    4943 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd08eb359_cf9e_422c_a3a7_35fd173ab7b6.slice/crio-846de2f8ed53650a03f7f87e3d7e5f2a34f8f6ac48ecd17c6dea4eeec20f9cd4 WatchSource:0}: Error finding container 846de2f8ed53650a03f7f87e3d7e5f2a34f8f6ac48ecd17c6dea4eeec20f9cd4: Status 404 returned error can't find the container with id 846de2f8ed53650a03f7f87e3d7e5f2a34f8f6ac48ecd17c6dea4eeec20f9cd4
Oct 30 13:16:22.657551 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:22.657505    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-glf25" event={"ID":"dbc82b1c-1c45-4263-b8ee-090fbecff086","Type":"ContainerStarted","Data":"08c663942fb9d9c7fa0105b756682ffc83a25223f84ca89191e85c5f7298e19c"}
Oct 30 13:16:22.657999 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:22.657576    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-glf25" event={"ID":"dbc82b1c-1c45-4263-b8ee-090fbecff086","Type":"ContainerStarted","Data":"e5ffcccf9ca8b7194cd890fe37f69da356a7be73f7ac72f5e477b8608f953777"}
Oct 30 13:16:22.659349 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:22.659319    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-05942d/distinct-w1-05942d-kserve-5fb44dc664-mcbkc" event={"ID":"d08eb359-cf9e-422c-a3a7-35fd173ab7b6","Type":"ContainerStarted","Data":"5c0097f27d893027b33c144a75796ecbad447f17006917035307e622e520f2d7"}
Oct 30 13:16:22.659448 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:22.659357    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-05942d/distinct-w1-05942d-kserve-5fb44dc664-mcbkc" event={"ID":"d08eb359-cf9e-422c-a3a7-35fd173ab7b6","Type":"ContainerStarted","Data":"846de2f8ed53650a03f7f87e3d7e5f2a34f8f6ac48ecd17c6dea4eeec20f9cd4"}
Oct 30 13:16:22.710192 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:22.710103    4943 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-glf25" podStartSLOduration=1.710088657 podStartE2EDuration="1.710088657s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:16:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:16:22.708524699 +0000 UTC m=+1158.496528429" watchObservedRunningTime="2026-10-30 13:16:22.710088657 +0000 UTC m=+1158.498092391"
Oct 30 13:16:22.755368 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:22.755328    4943 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w1-05942d/distinct-w1-05942d-kserve-5fb44dc664-mcbkc" podStartSLOduration=1.755315082 podStartE2EDuration="1.755315082s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:16:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:16:22.754692092 +0000 UTC m=+1158.542695822" watchObservedRunningTime="2026-10-30 13:16:22.755315082 +0000 UTC m=+1158.543318812"
Oct 30 13:16:22.919281 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:22.918862    4943 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-glf25"
Oct 30 13:16:23.128616 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:23.128540    4943 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-05942d/distinct-w1-05942d-kserve-5fb44dc664-mcbkc"
Oct 30 13:16:23.896416 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:23.896377    4943 patch_prober.go:29] interesting pod/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-glf25 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.90:15021/healthz/ready\": dial tcp 10.132.0.90:15021: connect: connection refused" start-of-body=
Oct 30 13:16:23.896853 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:23.896435    4943 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-glf25" podUID="dbc82b1c-1c45-4263-b8ee-090fbecff086" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.90:15021/healthz/ready\": dial tcp 10.132.0.90:15021: connect: connection refused"
Oct 30 13:16:24.894960 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:24.894917    4943 patch_prober.go:29] interesting pod/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-glf25 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.90:15021/healthz/ready\": dial tcp 10.132.0.90:15021: connect: connection refused" start-of-body=
Oct 30 13:16:24.895136 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:24.894986    4943 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-glf25" podUID="dbc82b1c-1c45-4263-b8ee-090fbecff086" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.90:15021/healthz/ready\": dial tcp 10.132.0.90:15021: connect: connection refused"
Oct 30 13:16:25.895309 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:25.895240    4943 patch_prober.go:29] interesting pod/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-glf25 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.90:15021/healthz/ready\": dial tcp 10.132.0.90:15021: connect: connection refused" start-of-body=
Oct 30 13:16:25.895721 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:25.895327    4943 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-glf25" podUID="dbc82b1c-1c45-4263-b8ee-090fbecff086" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.90:15021/healthz/ready\": dial tcp 10.132.0.90:15021: connect: connection refused"
Oct 30 13:16:26.419883 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:26.419851    4943 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w5-f8a55e/distinct-w5-f8a55e-kserve-894fbd994-s2vsp"]
Oct 30 13:16:26.424401 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:26.424373    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-f8a55e/distinct-w5-f8a55e-kserve-894fbd994-s2vsp"
Oct 30 13:16:26.442223 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:26.442205    4943 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w5-f8a55e\"/\"kube-root-ca.crt\""
Oct 30 13:16:26.443443 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:26.443424    4943 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w5-f8a55e\"/\"default-dockercfg-g4m6s\""
Oct 30 13:16:26.457509 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:26.457487    4943 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w5-f8a55e\"/\"distinct-w5-f8a55e-kserve-self-signed-certs\""
Oct 30 13:16:26.486279 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:26.462118    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w5-f8a55e/distinct-w5-f8a55e-kserve-894fbd994-s2vsp"]
Oct 30 13:16:26.513823 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:26.513791    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/30f54ef4-28c2-4bf2-9feb-e2b0f1232c9f-tls-certs\") pod \"distinct-w5-f8a55e-kserve-894fbd994-s2vsp\" (UID: \"30f54ef4-28c2-4bf2-9feb-e2b0f1232c9f\") " pod="e2e-models-e2e-worker-w5-f8a55e/distinct-w5-f8a55e-kserve-894fbd994-s2vsp"
Oct 30 13:16:26.513948 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:26.513831    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/30f54ef4-28c2-4bf2-9feb-e2b0f1232c9f-home\") pod \"distinct-w5-f8a55e-kserve-894fbd994-s2vsp\" (UID: \"30f54ef4-28c2-4bf2-9feb-e2b0f1232c9f\") " pod="e2e-models-e2e-worker-w5-f8a55e/distinct-w5-f8a55e-kserve-894fbd994-s2vsp"
Oct 30 13:16:26.513998 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:26.513978    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/30f54ef4-28c2-4bf2-9feb-e2b0f1232c9f-tmp-dir\") pod \"distinct-w5-f8a55e-kserve-894fbd994-s2vsp\" (UID: \"30f54ef4-28c2-4bf2-9feb-e2b0f1232c9f\") " pod="e2e-models-e2e-worker-w5-f8a55e/distinct-w5-f8a55e-kserve-894fbd994-s2vsp"
Oct 30 13:16:26.514114 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:26.514093    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k68z7\" (UniqueName: \"kubernetes.io/projected/30f54ef4-28c2-4bf2-9feb-e2b0f1232c9f-kube-api-access-k68z7\") pod \"distinct-w5-f8a55e-kserve-894fbd994-s2vsp\" (UID: \"30f54ef4-28c2-4bf2-9feb-e2b0f1232c9f\") " pod="e2e-models-e2e-worker-w5-f8a55e/distinct-w5-f8a55e-kserve-894fbd994-s2vsp"
Oct 30 13:16:26.514160 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:26.514147    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/30f54ef4-28c2-4bf2-9feb-e2b0f1232c9f-dshm\") pod \"distinct-w5-f8a55e-kserve-894fbd994-s2vsp\" (UID: \"30f54ef4-28c2-4bf2-9feb-e2b0f1232c9f\") " pod="e2e-models-e2e-worker-w5-f8a55e/distinct-w5-f8a55e-kserve-894fbd994-s2vsp"
Oct 30 13:16:26.514195 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:26.514178    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/30f54ef4-28c2-4bf2-9feb-e2b0f1232c9f-model-cache\") pod \"distinct-w5-f8a55e-kserve-894fbd994-s2vsp\" (UID: \"30f54ef4-28c2-4bf2-9feb-e2b0f1232c9f\") " pod="e2e-models-e2e-worker-w5-f8a55e/distinct-w5-f8a55e-kserve-894fbd994-s2vsp"
Oct 30 13:16:26.541980 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:26.541956    4943 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w5-f8a55e\"/\"openshift-service-ca.crt\""
Oct 30 13:16:26.616926 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:26.615129    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/30f54ef4-28c2-4bf2-9feb-e2b0f1232c9f-tmp-dir\") pod \"distinct-w5-f8a55e-kserve-894fbd994-s2vsp\" (UID: \"30f54ef4-28c2-4bf2-9feb-e2b0f1232c9f\") " pod="e2e-models-e2e-worker-w5-f8a55e/distinct-w5-f8a55e-kserve-894fbd994-s2vsp"
Oct 30 13:16:26.616926 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:26.615244    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k68z7\" (UniqueName: \"kubernetes.io/projected/30f54ef4-28c2-4bf2-9feb-e2b0f1232c9f-kube-api-access-k68z7\") pod \"distinct-w5-f8a55e-kserve-894fbd994-s2vsp\" (UID: \"30f54ef4-28c2-4bf2-9feb-e2b0f1232c9f\") " pod="e2e-models-e2e-worker-w5-f8a55e/distinct-w5-f8a55e-kserve-894fbd994-s2vsp"
Oct 30 13:16:26.616926 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:26.615323    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/30f54ef4-28c2-4bf2-9feb-e2b0f1232c9f-dshm\") pod \"distinct-w5-f8a55e-kserve-894fbd994-s2vsp\" (UID: \"30f54ef4-28c2-4bf2-9feb-e2b0f1232c9f\") " pod="e2e-models-e2e-worker-w5-f8a55e/distinct-w5-f8a55e-kserve-894fbd994-s2vsp"
Oct 30 13:16:26.616926 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:26.615364    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/30f54ef4-28c2-4bf2-9feb-e2b0f1232c9f-model-cache\") pod \"distinct-w5-f8a55e-kserve-894fbd994-s2vsp\" (UID: \"30f54ef4-28c2-4bf2-9feb-e2b0f1232c9f\") " pod="e2e-models-e2e-worker-w5-f8a55e/distinct-w5-f8a55e-kserve-894fbd994-s2vsp"
Oct 30 13:16:26.616926 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:26.615446    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/30f54ef4-28c2-4bf2-9feb-e2b0f1232c9f-tls-certs\") pod \"distinct-w5-f8a55e-kserve-894fbd994-s2vsp\" (UID: \"30f54ef4-28c2-4bf2-9feb-e2b0f1232c9f\") " pod="e2e-models-e2e-worker-w5-f8a55e/distinct-w5-f8a55e-kserve-894fbd994-s2vsp"
Oct 30 13:16:26.616926 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:26.615477    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/30f54ef4-28c2-4bf2-9feb-e2b0f1232c9f-home\") pod \"distinct-w5-f8a55e-kserve-894fbd994-s2vsp\" (UID: \"30f54ef4-28c2-4bf2-9feb-e2b0f1232c9f\") " pod="e2e-models-e2e-worker-w5-f8a55e/distinct-w5-f8a55e-kserve-894fbd994-s2vsp"
Oct 30 13:16:26.616926 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:26.616200    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/30f54ef4-28c2-4bf2-9feb-e2b0f1232c9f-home\") pod \"distinct-w5-f8a55e-kserve-894fbd994-s2vsp\" (UID: \"30f54ef4-28c2-4bf2-9feb-e2b0f1232c9f\") " pod="e2e-models-e2e-worker-w5-f8a55e/distinct-w5-f8a55e-kserve-894fbd994-s2vsp"
Oct 30 13:16:26.616926 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:26.616483    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/30f54ef4-28c2-4bf2-9feb-e2b0f1232c9f-tmp-dir\") pod \"distinct-w5-f8a55e-kserve-894fbd994-s2vsp\" (UID: \"30f54ef4-28c2-4bf2-9feb-e2b0f1232c9f\") " pod="e2e-models-e2e-worker-w5-f8a55e/distinct-w5-f8a55e-kserve-894fbd994-s2vsp"
Oct 30 13:16:26.617517 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:26.617343    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/30f54ef4-28c2-4bf2-9feb-e2b0f1232c9f-model-cache\") pod \"distinct-w5-f8a55e-kserve-894fbd994-s2vsp\" (UID: \"30f54ef4-28c2-4bf2-9feb-e2b0f1232c9f\") " pod="e2e-models-e2e-worker-w5-f8a55e/distinct-w5-f8a55e-kserve-894fbd994-s2vsp"
Oct 30 13:16:26.621417 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:26.621366    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/30f54ef4-28c2-4bf2-9feb-e2b0f1232c9f-tls-certs\") pod \"distinct-w5-f8a55e-kserve-894fbd994-s2vsp\" (UID: \"30f54ef4-28c2-4bf2-9feb-e2b0f1232c9f\") " pod="e2e-models-e2e-worker-w5-f8a55e/distinct-w5-f8a55e-kserve-894fbd994-s2vsp"
Oct 30 13:16:26.653993 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:26.653861    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/30f54ef4-28c2-4bf2-9feb-e2b0f1232c9f-dshm\") pod \"distinct-w5-f8a55e-kserve-894fbd994-s2vsp\" (UID: \"30f54ef4-28c2-4bf2-9feb-e2b0f1232c9f\") " pod="e2e-models-e2e-worker-w5-f8a55e/distinct-w5-f8a55e-kserve-894fbd994-s2vsp"
Oct 30 13:16:26.684334 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:26.678279    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k68z7\" (UniqueName: \"kubernetes.io/projected/30f54ef4-28c2-4bf2-9feb-e2b0f1232c9f-kube-api-access-k68z7\") pod \"distinct-w5-f8a55e-kserve-894fbd994-s2vsp\" (UID: \"30f54ef4-28c2-4bf2-9feb-e2b0f1232c9f\") " pod="e2e-models-e2e-worker-w5-f8a55e/distinct-w5-f8a55e-kserve-894fbd994-s2vsp"
Oct 30 13:16:26.743980 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:26.743923    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-f8a55e/distinct-w5-f8a55e-kserve-894fbd994-s2vsp"
Oct 30 13:16:26.894490 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:26.894237    4943 patch_prober.go:29] interesting pod/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-glf25 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.90:15021/healthz/ready\": dial tcp 10.132.0.90:15021: connect: connection refused" start-of-body=
Oct 30 13:16:26.894490 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:26.894335    4943 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-glf25" podUID="dbc82b1c-1c45-4263-b8ee-090fbecff086" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.90:15021/healthz/ready\": dial tcp 10.132.0.90:15021: connect: connection refused"
Oct 30 13:16:27.096302 ip-10-0-142-84 kubenswrapper[4943]: W1030 13:16:27.095704    4943 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod30f54ef4_28c2_4bf2_9feb_e2b0f1232c9f.slice/crio-0ddf785c8878aff5cca3b379282d3f4f62c2c2216850a7fba74a2e1e739e21bc WatchSource:0}: Error finding container 0ddf785c8878aff5cca3b379282d3f4f62c2c2216850a7fba74a2e1e739e21bc: Status 404 returned error can't find the container with id 0ddf785c8878aff5cca3b379282d3f4f62c2c2216850a7fba74a2e1e739e21bc
Oct 30 13:16:27.097001 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:27.096970    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w5-f8a55e/distinct-w5-f8a55e-kserve-894fbd994-s2vsp"]
Oct 30 13:16:27.753109 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:27.753071    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-f8a55e/distinct-w5-f8a55e-kserve-894fbd994-s2vsp" event={"ID":"30f54ef4-28c2-4bf2-9feb-e2b0f1232c9f","Type":"ContainerStarted","Data":"cb30fffcfc39cbf2406712457dc1353b9638d2c587563f00c7849722720d4823"}
Oct 30 13:16:27.753109 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:27.753119    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-f8a55e/distinct-w5-f8a55e-kserve-894fbd994-s2vsp" event={"ID":"30f54ef4-28c2-4bf2-9feb-e2b0f1232c9f","Type":"ContainerStarted","Data":"0ddf785c8878aff5cca3b379282d3f4f62c2c2216850a7fba74a2e1e739e21bc"}
Oct 30 13:16:27.802842 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:27.802785    4943 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w5-f8a55e/distinct-w5-f8a55e-kserve-894fbd994-s2vsp" podStartSLOduration=1.8027715309999999 podStartE2EDuration="1.802771531s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:16:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:16:27.801082702 +0000 UTC m=+1163.589086432" watchObservedRunningTime="2026-10-30 13:16:27.802771531 +0000 UTC m=+1163.590775274"
Oct 30 13:16:27.895013 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:27.894974    4943 patch_prober.go:29] interesting pod/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-glf25 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.90:15021/healthz/ready\": dial tcp 10.132.0.90:15021: connect: connection refused" start-of-body=
Oct 30 13:16:27.895181 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:27.895045    4943 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-glf25" podUID="dbc82b1c-1c45-4263-b8ee-090fbecff086" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.90:15021/healthz/ready\": dial tcp 10.132.0.90:15021: connect: connection refused"
Oct 30 13:16:28.756849 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:28.751271    4943 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-f8a55e/distinct-w5-f8a55e-kserve-894fbd994-s2vsp"
Oct 30 13:16:29.053787 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:29.053711    4943 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-glf25"
Oct 30 13:16:29.054801 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:29.054778    4943 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-glf25"
Oct 30 13:16:29.056397 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:29.056375    4943 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-glf25"
Oct 30 13:16:32.128872 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:32.128838    4943 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w1-05942d/distinct-w1-05942d-kserve-5fb44dc664-mcbkc"
Oct 30 13:16:32.162100 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:32.162068    4943 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w1-05942d/distinct-w1-05942d-kserve-5fb44dc664-mcbkc"
Oct 30 13:16:32.802175 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:32.802147    4943 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-05942d/distinct-w1-05942d-kserve-5fb44dc664-mcbkc"
Oct 30 13:16:33.761252 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:33.761216    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-hdmrm"]
Oct 30 13:16:36.099534 ip-10-0-142-84 kubenswrapper[4943]: E1030 13:16:36.099501    4943 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-knqz6" containerName="extractor"
Oct 30 13:16:36.745583 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:36.745546    4943 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w5-f8a55e/distinct-w5-f8a55e-kserve-894fbd994-s2vsp"
Oct 30 13:16:36.764456 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:36.764429    4943 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w5-f8a55e/distinct-w5-f8a55e-kserve-894fbd994-s2vsp"
Oct 30 13:16:36.876311 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:36.876281    4943 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-f8a55e/distinct-w5-f8a55e-kserve-894fbd994-s2vsp"
Oct 30 13:16:42.218980 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:42.218943    4943 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w3-d0a18c/distinct-2-w3-d0a18c-kserve-85f8ff4846-rc44r"]
Oct 30 13:16:42.224220 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:42.224199    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-d0a18c/distinct-2-w3-d0a18c-kserve-85f8ff4846-rc44r"
Oct 30 13:16:42.241454 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:42.241428    4943 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w3-d0a18c\"/\"distinct-2-w3-d0a18c-kserve-self-signed-certs\""
Oct 30 13:16:42.259562 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:42.259529    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w3-d0a18c/distinct-2-w3-d0a18c-kserve-85f8ff4846-rc44r"]
Oct 30 13:16:42.387337 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:42.387304    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j2rvq\" (UniqueName: \"kubernetes.io/projected/d65e47d2-4461-406a-9b71-bb3fac4262f0-kube-api-access-j2rvq\") pod \"distinct-2-w3-d0a18c-kserve-85f8ff4846-rc44r\" (UID: \"d65e47d2-4461-406a-9b71-bb3fac4262f0\") " pod="e2e-models-e2e-worker-w3-d0a18c/distinct-2-w3-d0a18c-kserve-85f8ff4846-rc44r"
Oct 30 13:16:42.387490 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:42.387414    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d65e47d2-4461-406a-9b71-bb3fac4262f0-tmp-dir\") pod \"distinct-2-w3-d0a18c-kserve-85f8ff4846-rc44r\" (UID: \"d65e47d2-4461-406a-9b71-bb3fac4262f0\") " pod="e2e-models-e2e-worker-w3-d0a18c/distinct-2-w3-d0a18c-kserve-85f8ff4846-rc44r"
Oct 30 13:16:42.387560 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:42.387545    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d65e47d2-4461-406a-9b71-bb3fac4262f0-tls-certs\") pod \"distinct-2-w3-d0a18c-kserve-85f8ff4846-rc44r\" (UID: \"d65e47d2-4461-406a-9b71-bb3fac4262f0\") " pod="e2e-models-e2e-worker-w3-d0a18c/distinct-2-w3-d0a18c-kserve-85f8ff4846-rc44r"
Oct 30 13:16:42.387599 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:42.387579    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d65e47d2-4461-406a-9b71-bb3fac4262f0-dshm\") pod \"distinct-2-w3-d0a18c-kserve-85f8ff4846-rc44r\" (UID: \"d65e47d2-4461-406a-9b71-bb3fac4262f0\") " pod="e2e-models-e2e-worker-w3-d0a18c/distinct-2-w3-d0a18c-kserve-85f8ff4846-rc44r"
Oct 30 13:16:42.387636 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:42.387621    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d65e47d2-4461-406a-9b71-bb3fac4262f0-model-cache\") pod \"distinct-2-w3-d0a18c-kserve-85f8ff4846-rc44r\" (UID: \"d65e47d2-4461-406a-9b71-bb3fac4262f0\") " pod="e2e-models-e2e-worker-w3-d0a18c/distinct-2-w3-d0a18c-kserve-85f8ff4846-rc44r"
Oct 30 13:16:42.387675 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:42.387663    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d65e47d2-4461-406a-9b71-bb3fac4262f0-home\") pod \"distinct-2-w3-d0a18c-kserve-85f8ff4846-rc44r\" (UID: \"d65e47d2-4461-406a-9b71-bb3fac4262f0\") " pod="e2e-models-e2e-worker-w3-d0a18c/distinct-2-w3-d0a18c-kserve-85f8ff4846-rc44r"
Oct 30 13:16:42.492334 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:42.490395    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d65e47d2-4461-406a-9b71-bb3fac4262f0-tmp-dir\") pod \"distinct-2-w3-d0a18c-kserve-85f8ff4846-rc44r\" (UID: \"d65e47d2-4461-406a-9b71-bb3fac4262f0\") " pod="e2e-models-e2e-worker-w3-d0a18c/distinct-2-w3-d0a18c-kserve-85f8ff4846-rc44r"
Oct 30 13:16:42.492334 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:42.490559    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d65e47d2-4461-406a-9b71-bb3fac4262f0-tls-certs\") pod \"distinct-2-w3-d0a18c-kserve-85f8ff4846-rc44r\" (UID: \"d65e47d2-4461-406a-9b71-bb3fac4262f0\") " pod="e2e-models-e2e-worker-w3-d0a18c/distinct-2-w3-d0a18c-kserve-85f8ff4846-rc44r"
Oct 30 13:16:42.492334 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:42.490602    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d65e47d2-4461-406a-9b71-bb3fac4262f0-dshm\") pod \"distinct-2-w3-d0a18c-kserve-85f8ff4846-rc44r\" (UID: \"d65e47d2-4461-406a-9b71-bb3fac4262f0\") " pod="e2e-models-e2e-worker-w3-d0a18c/distinct-2-w3-d0a18c-kserve-85f8ff4846-rc44r"
Oct 30 13:16:42.492334 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:42.490628    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d65e47d2-4461-406a-9b71-bb3fac4262f0-model-cache\") pod \"distinct-2-w3-d0a18c-kserve-85f8ff4846-rc44r\" (UID: \"d65e47d2-4461-406a-9b71-bb3fac4262f0\") " pod="e2e-models-e2e-worker-w3-d0a18c/distinct-2-w3-d0a18c-kserve-85f8ff4846-rc44r"
Oct 30 13:16:42.492334 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:42.490692    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d65e47d2-4461-406a-9b71-bb3fac4262f0-home\") pod \"distinct-2-w3-d0a18c-kserve-85f8ff4846-rc44r\" (UID: \"d65e47d2-4461-406a-9b71-bb3fac4262f0\") " pod="e2e-models-e2e-worker-w3-d0a18c/distinct-2-w3-d0a18c-kserve-85f8ff4846-rc44r"
Oct 30 13:16:42.492334 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:42.490752    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j2rvq\" (UniqueName: \"kubernetes.io/projected/d65e47d2-4461-406a-9b71-bb3fac4262f0-kube-api-access-j2rvq\") pod \"distinct-2-w3-d0a18c-kserve-85f8ff4846-rc44r\" (UID: \"d65e47d2-4461-406a-9b71-bb3fac4262f0\") " pod="e2e-models-e2e-worker-w3-d0a18c/distinct-2-w3-d0a18c-kserve-85f8ff4846-rc44r"
Oct 30 13:16:42.492334 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:42.491525    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d65e47d2-4461-406a-9b71-bb3fac4262f0-tmp-dir\") pod \"distinct-2-w3-d0a18c-kserve-85f8ff4846-rc44r\" (UID: \"d65e47d2-4461-406a-9b71-bb3fac4262f0\") " pod="e2e-models-e2e-worker-w3-d0a18c/distinct-2-w3-d0a18c-kserve-85f8ff4846-rc44r"
Oct 30 13:16:42.492334 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:42.492201    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d65e47d2-4461-406a-9b71-bb3fac4262f0-model-cache\") pod \"distinct-2-w3-d0a18c-kserve-85f8ff4846-rc44r\" (UID: \"d65e47d2-4461-406a-9b71-bb3fac4262f0\") " pod="e2e-models-e2e-worker-w3-d0a18c/distinct-2-w3-d0a18c-kserve-85f8ff4846-rc44r"
Oct 30 13:16:42.492871 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:42.492738    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d65e47d2-4461-406a-9b71-bb3fac4262f0-home\") pod \"distinct-2-w3-d0a18c-kserve-85f8ff4846-rc44r\" (UID: \"d65e47d2-4461-406a-9b71-bb3fac4262f0\") " pod="e2e-models-e2e-worker-w3-d0a18c/distinct-2-w3-d0a18c-kserve-85f8ff4846-rc44r"
Oct 30 13:16:42.499735 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:42.499709    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d65e47d2-4461-406a-9b71-bb3fac4262f0-tls-certs\") pod \"distinct-2-w3-d0a18c-kserve-85f8ff4846-rc44r\" (UID: \"d65e47d2-4461-406a-9b71-bb3fac4262f0\") " pod="e2e-models-e2e-worker-w3-d0a18c/distinct-2-w3-d0a18c-kserve-85f8ff4846-rc44r"
Oct 30 13:16:42.511644 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:42.511580    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d65e47d2-4461-406a-9b71-bb3fac4262f0-dshm\") pod \"distinct-2-w3-d0a18c-kserve-85f8ff4846-rc44r\" (UID: \"d65e47d2-4461-406a-9b71-bb3fac4262f0\") " pod="e2e-models-e2e-worker-w3-d0a18c/distinct-2-w3-d0a18c-kserve-85f8ff4846-rc44r"
Oct 30 13:16:42.518314 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:42.518283    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j2rvq\" (UniqueName: \"kubernetes.io/projected/d65e47d2-4461-406a-9b71-bb3fac4262f0-kube-api-access-j2rvq\") pod \"distinct-2-w3-d0a18c-kserve-85f8ff4846-rc44r\" (UID: \"d65e47d2-4461-406a-9b71-bb3fac4262f0\") " pod="e2e-models-e2e-worker-w3-d0a18c/distinct-2-w3-d0a18c-kserve-85f8ff4846-rc44r"
Oct 30 13:16:42.537842 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:42.537815    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-d0a18c/distinct-2-w3-d0a18c-kserve-85f8ff4846-rc44r"
Oct 30 13:16:42.846591 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:42.846557    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w3-d0a18c/distinct-2-w3-d0a18c-kserve-85f8ff4846-rc44r"]
Oct 30 13:16:42.846816 ip-10-0-142-84 kubenswrapper[4943]: W1030 13:16:42.846780    4943 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd65e47d2_4461_406a_9b71_bb3fac4262f0.slice/crio-60b0b5e7e17beaa1908e291825b35e65a78a7e7208c49375be251f2001742c8c WatchSource:0}: Error finding container 60b0b5e7e17beaa1908e291825b35e65a78a7e7208c49375be251f2001742c8c: Status 404 returned error can't find the container with id 60b0b5e7e17beaa1908e291825b35e65a78a7e7208c49375be251f2001742c8c
Oct 30 13:16:42.919581 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:42.919547    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-d0a18c/distinct-2-w3-d0a18c-kserve-85f8ff4846-rc44r" event={"ID":"d65e47d2-4461-406a-9b71-bb3fac4262f0","Type":"ContainerStarted","Data":"295b4280acd1ea46140fd906e618c2bee2291632e55178b166967be88fa38ce6"}
Oct 30 13:16:42.919692 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:42.919592    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-d0a18c/distinct-2-w3-d0a18c-kserve-85f8ff4846-rc44r" event={"ID":"d65e47d2-4461-406a-9b71-bb3fac4262f0","Type":"ContainerStarted","Data":"60b0b5e7e17beaa1908e291825b35e65a78a7e7208c49375be251f2001742c8c"}
Oct 30 13:16:42.972641 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:42.972590    4943 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w3-d0a18c/distinct-2-w3-d0a18c-kserve-85f8ff4846-rc44r" podStartSLOduration=0.972575075 podStartE2EDuration="972.575075ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:16:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:16:42.969586532 +0000 UTC m=+1178.757590276" watchObservedRunningTime="2026-10-30 13:16:42.972575075 +0000 UTC m=+1178.760578805"
Oct 30 13:16:43.538940 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:43.538906    4943 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-d0a18c/distinct-2-w3-d0a18c-kserve-85f8ff4846-rc44r"
Oct 30 13:16:45.230349 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:45.230314    4943 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-67065e92-qcnbs"]
Oct 30 13:16:45.234802 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:45.234779    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-67065e92-qcnbs"
Oct 30 13:16:45.255652 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:45.255624    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-67065e92-qcnbs"]
Oct 30 13:16:45.281316 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:45.265353    4943 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 13:16:45.281316 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:45.265641    4943 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-p9z57\""
Oct 30 13:16:45.341186 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:45.338001    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/3ab3da9a-ea4b-4638-996b-d1bf6a99a7d2-maas-api-xks-ca\") pod \"maas-api-revoke-sub-67065e92-qcnbs\" (UID: \"3ab3da9a-ea4b-4638-996b-d1bf6a99a7d2\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-67065e92-qcnbs"
Oct 30 13:16:45.341186 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:45.338059    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cz4rv\" (UniqueName: \"kubernetes.io/projected/3ab3da9a-ea4b-4638-996b-d1bf6a99a7d2-kube-api-access-cz4rv\") pod \"maas-api-revoke-sub-67065e92-qcnbs\" (UID: \"3ab3da9a-ea4b-4638-996b-d1bf6a99a7d2\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-67065e92-qcnbs"
Oct 30 13:16:45.341186 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:45.338094    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/3ab3da9a-ea4b-4638-996b-d1bf6a99a7d2-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-67065e92-qcnbs\" (UID: \"3ab3da9a-ea4b-4638-996b-d1bf6a99a7d2\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-67065e92-qcnbs"
Oct 30 13:16:45.443997 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:45.443964    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/3ab3da9a-ea4b-4638-996b-d1bf6a99a7d2-maas-api-xks-ca\") pod \"maas-api-revoke-sub-67065e92-qcnbs\" (UID: \"3ab3da9a-ea4b-4638-996b-d1bf6a99a7d2\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-67065e92-qcnbs"
Oct 30 13:16:45.443997 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:45.444000    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cz4rv\" (UniqueName: \"kubernetes.io/projected/3ab3da9a-ea4b-4638-996b-d1bf6a99a7d2-kube-api-access-cz4rv\") pod \"maas-api-revoke-sub-67065e92-qcnbs\" (UID: \"3ab3da9a-ea4b-4638-996b-d1bf6a99a7d2\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-67065e92-qcnbs"
Oct 30 13:16:45.444309 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:45.444045    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/3ab3da9a-ea4b-4638-996b-d1bf6a99a7d2-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-67065e92-qcnbs\" (UID: \"3ab3da9a-ea4b-4638-996b-d1bf6a99a7d2\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-67065e92-qcnbs"
Oct 30 13:16:45.444799 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:45.444777    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/3ab3da9a-ea4b-4638-996b-d1bf6a99a7d2-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-67065e92-qcnbs\" (UID: \"3ab3da9a-ea4b-4638-996b-d1bf6a99a7d2\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-67065e92-qcnbs"
Oct 30 13:16:45.446377 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:45.446355    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/3ab3da9a-ea4b-4638-996b-d1bf6a99a7d2-maas-api-xks-ca\") pod \"maas-api-revoke-sub-67065e92-qcnbs\" (UID: \"3ab3da9a-ea4b-4638-996b-d1bf6a99a7d2\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-67065e92-qcnbs"
Oct 30 13:16:45.469560 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:45.469534    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cz4rv\" (UniqueName: \"kubernetes.io/projected/3ab3da9a-ea4b-4638-996b-d1bf6a99a7d2-kube-api-access-cz4rv\") pod \"maas-api-revoke-sub-67065e92-qcnbs\" (UID: \"3ab3da9a-ea4b-4638-996b-d1bf6a99a7d2\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-67065e92-qcnbs"
Oct 30 13:16:45.549412 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:45.549342    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-67065e92-qcnbs"
Oct 30 13:16:45.869558 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:45.869525    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-67065e92-qcnbs"]
Oct 30 13:16:45.870032 ip-10-0-142-84 kubenswrapper[4943]: W1030 13:16:45.870000    4943 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3ab3da9a_ea4b_4638_996b_d1bf6a99a7d2.slice/crio-090702ca2d350396e6a709fdaac4ae0463df1a005e7a0690f3cc90609f4323dc WatchSource:0}: Error finding container 090702ca2d350396e6a709fdaac4ae0463df1a005e7a0690f3cc90609f4323dc: Status 404 returned error can't find the container with id 090702ca2d350396e6a709fdaac4ae0463df1a005e7a0690f3cc90609f4323dc
Oct 30 13:16:45.940740 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:45.940707    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-67065e92-qcnbs" event={"ID":"3ab3da9a-ea4b-4638-996b-d1bf6a99a7d2","Type":"ContainerStarted","Data":"090702ca2d350396e6a709fdaac4ae0463df1a005e7a0690f3cc90609f4323dc"}
Oct 30 13:16:46.952016 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:46.951924    4943 generic.go:356] "Generic (PLEG): container finished" podID="3ab3da9a-ea4b-4638-996b-d1bf6a99a7d2" containerID="44ec6a14c6426331b705f0e4714d5f0bfcb896e1313d09ca5ef7930ab3c0b11d" exitCode=0
Oct 30 13:16:46.952413 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:46.952010    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-67065e92-qcnbs" event={"ID":"3ab3da9a-ea4b-4638-996b-d1bf6a99a7d2","Type":"ContainerDied","Data":"44ec6a14c6426331b705f0e4714d5f0bfcb896e1313d09ca5ef7930ab3c0b11d"}
Oct 30 13:16:48.122368 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:48.122343    4943 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-67065e92-qcnbs"
Oct 30 13:16:48.283536 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:48.283449    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/3ab3da9a-ea4b-4638-996b-d1bf6a99a7d2-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/3ab3da9a-ea4b-4638-996b-d1bf6a99a7d2-maas-api-xks-ca\") pod \"3ab3da9a-ea4b-4638-996b-d1bf6a99a7d2\" (UID: \"3ab3da9a-ea4b-4638-996b-d1bf6a99a7d2\") "
Oct 30 13:16:48.283708 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:48.283541    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3ab3da9a-ea4b-4638-996b-d1bf6a99a7d2-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/3ab3da9a-ea4b-4638-996b-d1bf6a99a7d2-maas-api-openshift-ca\") pod \"3ab3da9a-ea4b-4638-996b-d1bf6a99a7d2\" (UID: \"3ab3da9a-ea4b-4638-996b-d1bf6a99a7d2\") "
Oct 30 13:16:48.283708 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:48.283653    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3ab3da9a-ea4b-4638-996b-d1bf6a99a7d2-kube-api-access-cz4rv\" (UniqueName: \"kubernetes.io/projected/3ab3da9a-ea4b-4638-996b-d1bf6a99a7d2-kube-api-access-cz4rv\") pod \"3ab3da9a-ea4b-4638-996b-d1bf6a99a7d2\" (UID: \"3ab3da9a-ea4b-4638-996b-d1bf6a99a7d2\") "
Oct 30 13:16:48.283953 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:48.283928    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3ab3da9a-ea4b-4638-996b-d1bf6a99a7d2-maas-api-openshift-ca" pod "3ab3da9a-ea4b-4638-996b-d1bf6a99a7d2" (UID: "3ab3da9a-ea4b-4638-996b-d1bf6a99a7d2"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 13:16:48.285655 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:48.285631    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3ab3da9a-ea4b-4638-996b-d1bf6a99a7d2-maas-api-xks-ca" pod "3ab3da9a-ea4b-4638-996b-d1bf6a99a7d2" (UID: "3ab3da9a-ea4b-4638-996b-d1bf6a99a7d2"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 13:16:48.285655 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:48.285634    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3ab3da9a-ea4b-4638-996b-d1bf6a99a7d2-kube-api-access-cz4rv" pod "3ab3da9a-ea4b-4638-996b-d1bf6a99a7d2" (UID: "3ab3da9a-ea4b-4638-996b-d1bf6a99a7d2"). InnerVolumeSpecName "kube-api-access-cz4rv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:16:48.385305 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:48.385249    4943 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/3ab3da9a-ea4b-4638-996b-d1bf6a99a7d2-maas-api-xks-ca\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:16:48.385408 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:48.385313    4943 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/3ab3da9a-ea4b-4638-996b-d1bf6a99a7d2-maas-api-openshift-ca\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:16:48.385408 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:48.385328    4943 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cz4rv\" (UniqueName: \"kubernetes.io/projected/3ab3da9a-ea4b-4638-996b-d1bf6a99a7d2-kube-api-access-cz4rv\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:16:48.967888 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:48.967858    4943 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-67065e92-qcnbs"
Oct 30 13:16:48.968076 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:48.967893    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-67065e92-qcnbs" event={"ID":"3ab3da9a-ea4b-4638-996b-d1bf6a99a7d2","Type":"ContainerDied","Data":"090702ca2d350396e6a709fdaac4ae0463df1a005e7a0690f3cc90609f4323dc"}
Oct 30 13:16:48.968076 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:48.967941    4943 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="090702ca2d350396e6a709fdaac4ae0463df1a005e7a0690f3cc90609f4323dc"
Oct 30 13:16:52.540279 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:52.540229    4943 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w3-d0a18c/distinct-2-w3-d0a18c-kserve-85f8ff4846-rc44r"
Oct 30 13:16:52.554489 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:52.554464    4943 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w3-d0a18c/distinct-2-w3-d0a18c-kserve-85f8ff4846-rc44r"
Oct 30 13:16:53.053283 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:53.051836    4943 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-d0a18c/distinct-2-w3-d0a18c-kserve-85f8ff4846-rc44r"
Oct 30 13:16:57.664854 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:57.664814    4943 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w1-05942d/distinct-2-w1-05942d-kserve-855c7f7c5f-pwt79"]
Oct 30 13:16:57.665508 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:57.665491    4943 cpu_manager.go:414] "containerMap: removing container" podUID="3ab3da9a-ea4b-4638-996b-d1bf6a99a7d2" containerName="revoke-keys"
Oct 30 13:16:57.665551 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:57.665510    4943 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3ab3da9a-ea4b-4638-996b-d1bf6a99a7d2" containerName="revoke-keys"
Oct 30 13:16:57.665650 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:57.665641    4943 memory_manager.go:365] "RemoveStaleState removing state" podUID="3ab3da9a-ea4b-4638-996b-d1bf6a99a7d2" containerName="revoke-keys"
Oct 30 13:16:57.669413 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:57.669395    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-05942d/distinct-2-w1-05942d-kserve-855c7f7c5f-pwt79"
Oct 30 13:16:57.737005 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:57.736967    4943 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w1-05942d\"/\"distinct-2-w1-05942d-kserve-self-signed-certs\""
Oct 30 13:16:57.742495 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:57.742454    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w1-05942d/distinct-2-w1-05942d-kserve-855c7f7c5f-pwt79"]
Oct 30 13:16:57.800276 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:57.800130    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3dac65dc-7a67-4fdb-840a-765235aaa5bf-home\") pod \"distinct-2-w1-05942d-kserve-855c7f7c5f-pwt79\" (UID: \"3dac65dc-7a67-4fdb-840a-765235aaa5bf\") " pod="e2e-models-e2e-worker-w1-05942d/distinct-2-w1-05942d-kserve-855c7f7c5f-pwt79"
Oct 30 13:16:57.801310 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:57.801277    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3dac65dc-7a67-4fdb-840a-765235aaa5bf-tmp-dir\") pod \"distinct-2-w1-05942d-kserve-855c7f7c5f-pwt79\" (UID: \"3dac65dc-7a67-4fdb-840a-765235aaa5bf\") " pod="e2e-models-e2e-worker-w1-05942d/distinct-2-w1-05942d-kserve-855c7f7c5f-pwt79"
Oct 30 13:16:57.801608 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:57.801588    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3dac65dc-7a67-4fdb-840a-765235aaa5bf-model-cache\") pod \"distinct-2-w1-05942d-kserve-855c7f7c5f-pwt79\" (UID: \"3dac65dc-7a67-4fdb-840a-765235aaa5bf\") " pod="e2e-models-e2e-worker-w1-05942d/distinct-2-w1-05942d-kserve-855c7f7c5f-pwt79"
Oct 30 13:16:57.801920 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:57.801902    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3dac65dc-7a67-4fdb-840a-765235aaa5bf-dshm\") pod \"distinct-2-w1-05942d-kserve-855c7f7c5f-pwt79\" (UID: \"3dac65dc-7a67-4fdb-840a-765235aaa5bf\") " pod="e2e-models-e2e-worker-w1-05942d/distinct-2-w1-05942d-kserve-855c7f7c5f-pwt79"
Oct 30 13:16:57.802198 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:57.802142    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jwtt4\" (UniqueName: \"kubernetes.io/projected/3dac65dc-7a67-4fdb-840a-765235aaa5bf-kube-api-access-jwtt4\") pod \"distinct-2-w1-05942d-kserve-855c7f7c5f-pwt79\" (UID: \"3dac65dc-7a67-4fdb-840a-765235aaa5bf\") " pod="e2e-models-e2e-worker-w1-05942d/distinct-2-w1-05942d-kserve-855c7f7c5f-pwt79"
Oct 30 13:16:57.802407 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:57.802387    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3dac65dc-7a67-4fdb-840a-765235aaa5bf-tls-certs\") pod \"distinct-2-w1-05942d-kserve-855c7f7c5f-pwt79\" (UID: \"3dac65dc-7a67-4fdb-840a-765235aaa5bf\") " pod="e2e-models-e2e-worker-w1-05942d/distinct-2-w1-05942d-kserve-855c7f7c5f-pwt79"
Oct 30 13:16:57.905190 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:57.905135    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3dac65dc-7a67-4fdb-840a-765235aaa5bf-tmp-dir\") pod \"distinct-2-w1-05942d-kserve-855c7f7c5f-pwt79\" (UID: \"3dac65dc-7a67-4fdb-840a-765235aaa5bf\") " pod="e2e-models-e2e-worker-w1-05942d/distinct-2-w1-05942d-kserve-855c7f7c5f-pwt79"
Oct 30 13:16:57.906024 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:57.905998    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3dac65dc-7a67-4fdb-840a-765235aaa5bf-model-cache\") pod \"distinct-2-w1-05942d-kserve-855c7f7c5f-pwt79\" (UID: \"3dac65dc-7a67-4fdb-840a-765235aaa5bf\") " pod="e2e-models-e2e-worker-w1-05942d/distinct-2-w1-05942d-kserve-855c7f7c5f-pwt79"
Oct 30 13:16:57.906624 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:57.906603    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3dac65dc-7a67-4fdb-840a-765235aaa5bf-dshm\") pod \"distinct-2-w1-05942d-kserve-855c7f7c5f-pwt79\" (UID: \"3dac65dc-7a67-4fdb-840a-765235aaa5bf\") " pod="e2e-models-e2e-worker-w1-05942d/distinct-2-w1-05942d-kserve-855c7f7c5f-pwt79"
Oct 30 13:16:57.917328 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:57.907092    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jwtt4\" (UniqueName: \"kubernetes.io/projected/3dac65dc-7a67-4fdb-840a-765235aaa5bf-kube-api-access-jwtt4\") pod \"distinct-2-w1-05942d-kserve-855c7f7c5f-pwt79\" (UID: \"3dac65dc-7a67-4fdb-840a-765235aaa5bf\") " pod="e2e-models-e2e-worker-w1-05942d/distinct-2-w1-05942d-kserve-855c7f7c5f-pwt79"
Oct 30 13:16:57.917711 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:57.917683    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3dac65dc-7a67-4fdb-840a-765235aaa5bf-tls-certs\") pod \"distinct-2-w1-05942d-kserve-855c7f7c5f-pwt79\" (UID: \"3dac65dc-7a67-4fdb-840a-765235aaa5bf\") " pod="e2e-models-e2e-worker-w1-05942d/distinct-2-w1-05942d-kserve-855c7f7c5f-pwt79"
Oct 30 13:16:57.918463 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:57.918440    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3dac65dc-7a67-4fdb-840a-765235aaa5bf-home\") pod \"distinct-2-w1-05942d-kserve-855c7f7c5f-pwt79\" (UID: \"3dac65dc-7a67-4fdb-840a-765235aaa5bf\") " pod="e2e-models-e2e-worker-w1-05942d/distinct-2-w1-05942d-kserve-855c7f7c5f-pwt79"
Oct 30 13:16:57.918787 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:57.905922    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3dac65dc-7a67-4fdb-840a-765235aaa5bf-tmp-dir\") pod \"distinct-2-w1-05942d-kserve-855c7f7c5f-pwt79\" (UID: \"3dac65dc-7a67-4fdb-840a-765235aaa5bf\") " pod="e2e-models-e2e-worker-w1-05942d/distinct-2-w1-05942d-kserve-855c7f7c5f-pwt79"
Oct 30 13:16:57.918903 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:57.906435    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3dac65dc-7a67-4fdb-840a-765235aaa5bf-model-cache\") pod \"distinct-2-w1-05942d-kserve-855c7f7c5f-pwt79\" (UID: \"3dac65dc-7a67-4fdb-840a-765235aaa5bf\") " pod="e2e-models-e2e-worker-w1-05942d/distinct-2-w1-05942d-kserve-855c7f7c5f-pwt79"
Oct 30 13:16:57.934923 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:57.919292    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3dac65dc-7a67-4fdb-840a-765235aaa5bf-home\") pod \"distinct-2-w1-05942d-kserve-855c7f7c5f-pwt79\" (UID: \"3dac65dc-7a67-4fdb-840a-765235aaa5bf\") " pod="e2e-models-e2e-worker-w1-05942d/distinct-2-w1-05942d-kserve-855c7f7c5f-pwt79"
Oct 30 13:16:57.935206 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:57.929318    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3dac65dc-7a67-4fdb-840a-765235aaa5bf-tls-certs\") pod \"distinct-2-w1-05942d-kserve-855c7f7c5f-pwt79\" (UID: \"3dac65dc-7a67-4fdb-840a-765235aaa5bf\") " pod="e2e-models-e2e-worker-w1-05942d/distinct-2-w1-05942d-kserve-855c7f7c5f-pwt79"
Oct 30 13:16:57.956288 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:57.955098    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3dac65dc-7a67-4fdb-840a-765235aaa5bf-dshm\") pod \"distinct-2-w1-05942d-kserve-855c7f7c5f-pwt79\" (UID: \"3dac65dc-7a67-4fdb-840a-765235aaa5bf\") " pod="e2e-models-e2e-worker-w1-05942d/distinct-2-w1-05942d-kserve-855c7f7c5f-pwt79"
Oct 30 13:16:57.974283 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:57.971350    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jwtt4\" (UniqueName: \"kubernetes.io/projected/3dac65dc-7a67-4fdb-840a-765235aaa5bf-kube-api-access-jwtt4\") pod \"distinct-2-w1-05942d-kserve-855c7f7c5f-pwt79\" (UID: \"3dac65dc-7a67-4fdb-840a-765235aaa5bf\") " pod="e2e-models-e2e-worker-w1-05942d/distinct-2-w1-05942d-kserve-855c7f7c5f-pwt79"
Oct 30 13:16:57.983485 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:57.983102    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-05942d/distinct-2-w1-05942d-kserve-855c7f7c5f-pwt79"
Oct 30 13:16:58.309323 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:58.309288    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w1-05942d/distinct-2-w1-05942d-kserve-855c7f7c5f-pwt79"]
Oct 30 13:16:58.309565 ip-10-0-142-84 kubenswrapper[4943]: W1030 13:16:58.309543    4943 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3dac65dc_7a67_4fdb_840a_765235aaa5bf.slice/crio-26f0ed8486078bf57adf8b457209769b4addd1a3ccf9e15c445b9fbb3fdb64c7 WatchSource:0}: Error finding container 26f0ed8486078bf57adf8b457209769b4addd1a3ccf9e15c445b9fbb3fdb64c7: Status 404 returned error can't find the container with id 26f0ed8486078bf57adf8b457209769b4addd1a3ccf9e15c445b9fbb3fdb64c7
Oct 30 13:16:59.146387 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:59.139449    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-05942d/distinct-2-w1-05942d-kserve-855c7f7c5f-pwt79" event={"ID":"3dac65dc-7a67-4fdb-840a-765235aaa5bf","Type":"ContainerStarted","Data":"632ff98e6b4cf4c86974d5cbbb1124f5f9e7eeacca5c2759c012e3e910264255"}
Oct 30 13:16:59.146387 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:59.139505    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-05942d/distinct-2-w1-05942d-kserve-855c7f7c5f-pwt79" event={"ID":"3dac65dc-7a67-4fdb-840a-765235aaa5bf","Type":"ContainerStarted","Data":"26f0ed8486078bf57adf8b457209769b4addd1a3ccf9e15c445b9fbb3fdb64c7"}
Oct 30 13:16:59.208078 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:59.208018    4943 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w1-05942d/distinct-2-w1-05942d-kserve-855c7f7c5f-pwt79" podStartSLOduration=2.208004133 podStartE2EDuration="2.208004133s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:16:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:16:59.205786139 +0000 UTC m=+1194.993789868" watchObservedRunningTime="2026-10-30 13:16:59.208004133 +0000 UTC m=+1194.996007863"
Oct 30 13:16:59.984160 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:16:59.984125    4943 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-05942d/distinct-2-w1-05942d-kserve-855c7f7c5f-pwt79"
Oct 30 13:17:02.827894 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:02.827849    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-hdmrm"]
Oct 30 13:17:04.481456 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:04.481421    4943 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ajkfrv_17f5a8d2-67e0-4489-8734-00f0d4483788/extract/0.log"
Oct 30 13:17:04.487406 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:04.487382    4943 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ajkfrv_17f5a8d2-67e0-4489-8734-00f0d4483788/extract/0.log"
Oct 30 13:17:04.496787 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:04.496764    4943 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-8fqfr_d8930ee7-0ae0-48b5-8ce3-cb8820355cef/console-operator/2.log"
Oct 30 13:17:04.503422 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:04.503400    4943 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-8fqfr_d8930ee7-0ae0-48b5-8ce3-cb8820355cef/console-operator/2.log"
Oct 30 13:17:05.955188 ip-10-0-142-84 kubenswrapper[4943]: E1030 13:17:05.955151    4943 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-knqz6" containerName="extractor"
Oct 30 13:17:06.438193 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:06.438162    4943 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-w2nsg"]
Oct 30 13:17:06.442697 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:06.442677    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-w2nsg"
Oct 30 13:17:06.472209 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:06.472178    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-w2nsg"]
Oct 30 13:17:06.533937 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:06.533905    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/558c9168-9f89-44f8-845b-74fe5345386c-workload-socket\") pod \"e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-w2nsg\" (UID: \"558c9168-9f89-44f8-845b-74fe5345386c\") " pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-w2nsg"
Oct 30 13:17:06.534100 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:06.533975    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/558c9168-9f89-44f8-845b-74fe5345386c-workload-certs\") pod \"e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-w2nsg\" (UID: \"558c9168-9f89-44f8-845b-74fe5345386c\") " pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-w2nsg"
Oct 30 13:17:06.534100 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:06.534062    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/558c9168-9f89-44f8-845b-74fe5345386c-istiod-ca-cert\") pod \"e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-w2nsg\" (UID: \"558c9168-9f89-44f8-845b-74fe5345386c\") " pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-w2nsg"
Oct 30 13:17:06.534211 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:06.534191    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/558c9168-9f89-44f8-845b-74fe5345386c-istio-envoy\") pod \"e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-w2nsg\" (UID: \"558c9168-9f89-44f8-845b-74fe5345386c\") " pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-w2nsg"
Oct 30 13:17:06.534310 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:06.534218    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/558c9168-9f89-44f8-845b-74fe5345386c-istio-podinfo\") pod \"e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-w2nsg\" (UID: \"558c9168-9f89-44f8-845b-74fe5345386c\") " pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-w2nsg"
Oct 30 13:17:06.534310 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:06.534305    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-94h9j\" (UniqueName: \"kubernetes.io/projected/558c9168-9f89-44f8-845b-74fe5345386c-kube-api-access-94h9j\") pod \"e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-w2nsg\" (UID: \"558c9168-9f89-44f8-845b-74fe5345386c\") " pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-w2nsg"
Oct 30 13:17:06.534404 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:06.534341    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/558c9168-9f89-44f8-845b-74fe5345386c-istio-token\") pod \"e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-w2nsg\" (UID: \"558c9168-9f89-44f8-845b-74fe5345386c\") " pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-w2nsg"
Oct 30 13:17:06.534404 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:06.534364    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/558c9168-9f89-44f8-845b-74fe5345386c-istio-data\") pod \"e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-w2nsg\" (UID: \"558c9168-9f89-44f8-845b-74fe5345386c\") " pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-w2nsg"
Oct 30 13:17:06.534502 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:06.534479    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/558c9168-9f89-44f8-845b-74fe5345386c-credential-socket\") pod \"e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-w2nsg\" (UID: \"558c9168-9f89-44f8-845b-74fe5345386c\") " pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-w2nsg"
Oct 30 13:17:06.636046 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:06.636009    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/558c9168-9f89-44f8-845b-74fe5345386c-istio-token\") pod \"e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-w2nsg\" (UID: \"558c9168-9f89-44f8-845b-74fe5345386c\") " pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-w2nsg"
Oct 30 13:17:06.636191 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:06.636052    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/558c9168-9f89-44f8-845b-74fe5345386c-istio-data\") pod \"e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-w2nsg\" (UID: \"558c9168-9f89-44f8-845b-74fe5345386c\") " pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-w2nsg"
Oct 30 13:17:06.636191 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:06.636091    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/558c9168-9f89-44f8-845b-74fe5345386c-credential-socket\") pod \"e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-w2nsg\" (UID: \"558c9168-9f89-44f8-845b-74fe5345386c\") " pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-w2nsg"
Oct 30 13:17:06.636191 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:06.636167    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/558c9168-9f89-44f8-845b-74fe5345386c-workload-socket\") pod \"e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-w2nsg\" (UID: \"558c9168-9f89-44f8-845b-74fe5345386c\") " pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-w2nsg"
Oct 30 13:17:06.636391 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:06.636222    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/558c9168-9f89-44f8-845b-74fe5345386c-workload-certs\") pod \"e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-w2nsg\" (UID: \"558c9168-9f89-44f8-845b-74fe5345386c\") " pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-w2nsg"
Oct 30 13:17:06.636391 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:06.636302    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/558c9168-9f89-44f8-845b-74fe5345386c-istiod-ca-cert\") pod \"e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-w2nsg\" (UID: \"558c9168-9f89-44f8-845b-74fe5345386c\") " pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-w2nsg"
Oct 30 13:17:06.636488 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:06.636419    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/558c9168-9f89-44f8-845b-74fe5345386c-istio-envoy\") pod \"e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-w2nsg\" (UID: \"558c9168-9f89-44f8-845b-74fe5345386c\") " pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-w2nsg"
Oct 30 13:17:06.636488 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:06.636443    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/558c9168-9f89-44f8-845b-74fe5345386c-istio-podinfo\") pod \"e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-w2nsg\" (UID: \"558c9168-9f89-44f8-845b-74fe5345386c\") " pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-w2nsg"
Oct 30 13:17:06.636488 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:06.636478    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-94h9j\" (UniqueName: \"kubernetes.io/projected/558c9168-9f89-44f8-845b-74fe5345386c-kube-api-access-94h9j\") pod \"e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-w2nsg\" (UID: \"558c9168-9f89-44f8-845b-74fe5345386c\") " pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-w2nsg"
Oct 30 13:17:06.636645 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:06.636527    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/558c9168-9f89-44f8-845b-74fe5345386c-istio-data\") pod \"e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-w2nsg\" (UID: \"558c9168-9f89-44f8-845b-74fe5345386c\") " pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-w2nsg"
Oct 30 13:17:06.636645 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:06.636563    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/558c9168-9f89-44f8-845b-74fe5345386c-credential-socket\") pod \"e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-w2nsg\" (UID: \"558c9168-9f89-44f8-845b-74fe5345386c\") " pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-w2nsg"
Oct 30 13:17:06.636752 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:06.636639    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/558c9168-9f89-44f8-845b-74fe5345386c-workload-certs\") pod \"e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-w2nsg\" (UID: \"558c9168-9f89-44f8-845b-74fe5345386c\") " pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-w2nsg"
Oct 30 13:17:06.636752 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:06.636674    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/558c9168-9f89-44f8-845b-74fe5345386c-workload-socket\") pod \"e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-w2nsg\" (UID: \"558c9168-9f89-44f8-845b-74fe5345386c\") " pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-w2nsg"
Oct 30 13:17:06.637098 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:06.637073    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/558c9168-9f89-44f8-845b-74fe5345386c-istiod-ca-cert\") pod \"e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-w2nsg\" (UID: \"558c9168-9f89-44f8-845b-74fe5345386c\") " pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-w2nsg"
Oct 30 13:17:06.638886 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:06.638864    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/558c9168-9f89-44f8-845b-74fe5345386c-istio-envoy\") pod \"e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-w2nsg\" (UID: \"558c9168-9f89-44f8-845b-74fe5345386c\") " pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-w2nsg"
Oct 30 13:17:06.639037 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:06.639018    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/558c9168-9f89-44f8-845b-74fe5345386c-istio-podinfo\") pod \"e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-w2nsg\" (UID: \"558c9168-9f89-44f8-845b-74fe5345386c\") " pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-w2nsg"
Oct 30 13:17:06.660955 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:06.660930    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/558c9168-9f89-44f8-845b-74fe5345386c-istio-token\") pod \"e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-w2nsg\" (UID: \"558c9168-9f89-44f8-845b-74fe5345386c\") " pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-w2nsg"
Oct 30 13:17:06.662959 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:06.662939    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-94h9j\" (UniqueName: \"kubernetes.io/projected/558c9168-9f89-44f8-845b-74fe5345386c-kube-api-access-94h9j\") pod \"e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-w2nsg\" (UID: \"558c9168-9f89-44f8-845b-74fe5345386c\") " pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-w2nsg"
Oct 30 13:17:06.758028 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:06.757952    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-w2nsg"
Oct 30 13:17:06.973881 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:06.973842    4943 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-t7zm7"]
Oct 30 13:17:06.979302 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:06.979254    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-t7zm7"
Oct 30 13:17:07.013397 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:07.013332    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-t7zm7"]
Oct 30 13:17:07.041927 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:07.041895    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0ee4232a-d7fe-4aa9-963c-4bac5daa86c3-istio-data\") pod \"e2e-worker-w3-d0a18c-openshift-default-547d57db97-t7zm7\" (UID: \"0ee4232a-d7fe-4aa9-963c-4bac5daa86c3\") " pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-t7zm7"
Oct 30 13:17:07.042105 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:07.041934    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6cvkg\" (UniqueName: \"kubernetes.io/projected/0ee4232a-d7fe-4aa9-963c-4bac5daa86c3-kube-api-access-6cvkg\") pod \"e2e-worker-w3-d0a18c-openshift-default-547d57db97-t7zm7\" (UID: \"0ee4232a-d7fe-4aa9-963c-4bac5daa86c3\") " pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-t7zm7"
Oct 30 13:17:07.042105 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:07.041968    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0ee4232a-d7fe-4aa9-963c-4bac5daa86c3-istio-envoy\") pod \"e2e-worker-w3-d0a18c-openshift-default-547d57db97-t7zm7\" (UID: \"0ee4232a-d7fe-4aa9-963c-4bac5daa86c3\") " pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-t7zm7"
Oct 30 13:17:07.042105 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:07.042040    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0ee4232a-d7fe-4aa9-963c-4bac5daa86c3-istiod-ca-cert\") pod \"e2e-worker-w3-d0a18c-openshift-default-547d57db97-t7zm7\" (UID: \"0ee4232a-d7fe-4aa9-963c-4bac5daa86c3\") " pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-t7zm7"
Oct 30 13:17:07.042299 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:07.042154    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0ee4232a-d7fe-4aa9-963c-4bac5daa86c3-workload-socket\") pod \"e2e-worker-w3-d0a18c-openshift-default-547d57db97-t7zm7\" (UID: \"0ee4232a-d7fe-4aa9-963c-4bac5daa86c3\") " pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-t7zm7"
Oct 30 13:17:07.042299 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:07.042205    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0ee4232a-d7fe-4aa9-963c-4bac5daa86c3-istio-token\") pod \"e2e-worker-w3-d0a18c-openshift-default-547d57db97-t7zm7\" (UID: \"0ee4232a-d7fe-4aa9-963c-4bac5daa86c3\") " pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-t7zm7"
Oct 30 13:17:07.042376 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:07.042310    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0ee4232a-d7fe-4aa9-963c-4bac5daa86c3-credential-socket\") pod \"e2e-worker-w3-d0a18c-openshift-default-547d57db97-t7zm7\" (UID: \"0ee4232a-d7fe-4aa9-963c-4bac5daa86c3\") " pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-t7zm7"
Oct 30 13:17:07.042376 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:07.042362    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0ee4232a-d7fe-4aa9-963c-4bac5daa86c3-istio-podinfo\") pod \"e2e-worker-w3-d0a18c-openshift-default-547d57db97-t7zm7\" (UID: \"0ee4232a-d7fe-4aa9-963c-4bac5daa86c3\") " pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-t7zm7"
Oct 30 13:17:07.042446 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:07.042391    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0ee4232a-d7fe-4aa9-963c-4bac5daa86c3-workload-certs\") pod \"e2e-worker-w3-d0a18c-openshift-default-547d57db97-t7zm7\" (UID: \"0ee4232a-d7fe-4aa9-963c-4bac5daa86c3\") " pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-t7zm7"
Oct 30 13:17:07.103408 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:07.103378    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-w2nsg"]
Oct 30 13:17:07.103526 ip-10-0-142-84 kubenswrapper[4943]: W1030 13:17:07.103405    4943 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod558c9168_9f89_44f8_845b_74fe5345386c.slice/crio-18b4fee8f9edc8f1be6601214884ed6202b2a186ecef5b89dfabdf3f6a92bfff WatchSource:0}: Error finding container 18b4fee8f9edc8f1be6601214884ed6202b2a186ecef5b89dfabdf3f6a92bfff: Status 404 returned error can't find the container with id 18b4fee8f9edc8f1be6601214884ed6202b2a186ecef5b89dfabdf3f6a92bfff
Oct 30 13:17:07.106164 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:07.106133    4943 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 13:17:07.106247 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:07.106195    4943 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 13:17:07.106247 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:07.106227    4943 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 13:17:07.144020 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:07.143996    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0ee4232a-d7fe-4aa9-963c-4bac5daa86c3-istio-data\") pod \"e2e-worker-w3-d0a18c-openshift-default-547d57db97-t7zm7\" (UID: \"0ee4232a-d7fe-4aa9-963c-4bac5daa86c3\") " pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-t7zm7"
Oct 30 13:17:07.144112 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:07.144031    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6cvkg\" (UniqueName: \"kubernetes.io/projected/0ee4232a-d7fe-4aa9-963c-4bac5daa86c3-kube-api-access-6cvkg\") pod \"e2e-worker-w3-d0a18c-openshift-default-547d57db97-t7zm7\" (UID: \"0ee4232a-d7fe-4aa9-963c-4bac5daa86c3\") " pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-t7zm7"
Oct 30 13:17:07.144112 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:07.144060    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0ee4232a-d7fe-4aa9-963c-4bac5daa86c3-istio-envoy\") pod \"e2e-worker-w3-d0a18c-openshift-default-547d57db97-t7zm7\" (UID: \"0ee4232a-d7fe-4aa9-963c-4bac5daa86c3\") " pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-t7zm7"
Oct 30 13:17:07.144235 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:07.144185    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0ee4232a-d7fe-4aa9-963c-4bac5daa86c3-istiod-ca-cert\") pod \"e2e-worker-w3-d0a18c-openshift-default-547d57db97-t7zm7\" (UID: \"0ee4232a-d7fe-4aa9-963c-4bac5daa86c3\") " pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-t7zm7"
Oct 30 13:17:07.144309 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:07.144247    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0ee4232a-d7fe-4aa9-963c-4bac5daa86c3-workload-socket\") pod \"e2e-worker-w3-d0a18c-openshift-default-547d57db97-t7zm7\" (UID: \"0ee4232a-d7fe-4aa9-963c-4bac5daa86c3\") " pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-t7zm7"
Oct 30 13:17:07.144367 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:07.144310    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0ee4232a-d7fe-4aa9-963c-4bac5daa86c3-istio-token\") pod \"e2e-worker-w3-d0a18c-openshift-default-547d57db97-t7zm7\" (UID: \"0ee4232a-d7fe-4aa9-963c-4bac5daa86c3\") " pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-t7zm7"
Oct 30 13:17:07.144419 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:07.144361    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0ee4232a-d7fe-4aa9-963c-4bac5daa86c3-credential-socket\") pod \"e2e-worker-w3-d0a18c-openshift-default-547d57db97-t7zm7\" (UID: \"0ee4232a-d7fe-4aa9-963c-4bac5daa86c3\") " pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-t7zm7"
Oct 30 13:17:07.144486 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:07.144420    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0ee4232a-d7fe-4aa9-963c-4bac5daa86c3-istio-podinfo\") pod \"e2e-worker-w3-d0a18c-openshift-default-547d57db97-t7zm7\" (UID: \"0ee4232a-d7fe-4aa9-963c-4bac5daa86c3\") " pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-t7zm7"
Oct 30 13:17:07.144486 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:07.144465    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0ee4232a-d7fe-4aa9-963c-4bac5daa86c3-workload-certs\") pod \"e2e-worker-w3-d0a18c-openshift-default-547d57db97-t7zm7\" (UID: \"0ee4232a-d7fe-4aa9-963c-4bac5daa86c3\") " pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-t7zm7"
Oct 30 13:17:07.144589 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:07.144492    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0ee4232a-d7fe-4aa9-963c-4bac5daa86c3-istio-data\") pod \"e2e-worker-w3-d0a18c-openshift-default-547d57db97-t7zm7\" (UID: \"0ee4232a-d7fe-4aa9-963c-4bac5daa86c3\") " pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-t7zm7"
Oct 30 13:17:07.144775 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:07.144754    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0ee4232a-d7fe-4aa9-963c-4bac5daa86c3-workload-certs\") pod \"e2e-worker-w3-d0a18c-openshift-default-547d57db97-t7zm7\" (UID: \"0ee4232a-d7fe-4aa9-963c-4bac5daa86c3\") " pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-t7zm7"
Oct 30 13:17:07.144843 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:07.144771    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0ee4232a-d7fe-4aa9-963c-4bac5daa86c3-workload-socket\") pod \"e2e-worker-w3-d0a18c-openshift-default-547d57db97-t7zm7\" (UID: \"0ee4232a-d7fe-4aa9-963c-4bac5daa86c3\") " pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-t7zm7"
Oct 30 13:17:07.144843 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:07.144803    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0ee4232a-d7fe-4aa9-963c-4bac5daa86c3-credential-socket\") pod \"e2e-worker-w3-d0a18c-openshift-default-547d57db97-t7zm7\" (UID: \"0ee4232a-d7fe-4aa9-963c-4bac5daa86c3\") " pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-t7zm7"
Oct 30 13:17:07.144964 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:07.144903    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0ee4232a-d7fe-4aa9-963c-4bac5daa86c3-istiod-ca-cert\") pod \"e2e-worker-w3-d0a18c-openshift-default-547d57db97-t7zm7\" (UID: \"0ee4232a-d7fe-4aa9-963c-4bac5daa86c3\") " pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-t7zm7"
Oct 30 13:17:07.146819 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:07.146795    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0ee4232a-d7fe-4aa9-963c-4bac5daa86c3-istio-envoy\") pod \"e2e-worker-w3-d0a18c-openshift-default-547d57db97-t7zm7\" (UID: \"0ee4232a-d7fe-4aa9-963c-4bac5daa86c3\") " pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-t7zm7"
Oct 30 13:17:07.147120 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:07.147100    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0ee4232a-d7fe-4aa9-963c-4bac5daa86c3-istio-podinfo\") pod \"e2e-worker-w3-d0a18c-openshift-default-547d57db97-t7zm7\" (UID: \"0ee4232a-d7fe-4aa9-963c-4bac5daa86c3\") " pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-t7zm7"
Oct 30 13:17:07.174341 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:07.174315    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6cvkg\" (UniqueName: \"kubernetes.io/projected/0ee4232a-d7fe-4aa9-963c-4bac5daa86c3-kube-api-access-6cvkg\") pod \"e2e-worker-w3-d0a18c-openshift-default-547d57db97-t7zm7\" (UID: \"0ee4232a-d7fe-4aa9-963c-4bac5daa86c3\") " pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-t7zm7"
Oct 30 13:17:07.175852 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:07.175829    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0ee4232a-d7fe-4aa9-963c-4bac5daa86c3-istio-token\") pod \"e2e-worker-w3-d0a18c-openshift-default-547d57db97-t7zm7\" (UID: \"0ee4232a-d7fe-4aa9-963c-4bac5daa86c3\") " pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-t7zm7"
Oct 30 13:17:07.270602 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:07.270507    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-w2nsg" event={"ID":"558c9168-9f89-44f8-845b-74fe5345386c","Type":"ContainerStarted","Data":"e4527f5cb84ce7b0edaa823d9748c98c31f60d8aca23cc89ef52010097eedfe8"}
Oct 30 13:17:07.270602 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:07.270574    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-w2nsg" event={"ID":"558c9168-9f89-44f8-845b-74fe5345386c","Type":"ContainerStarted","Data":"18b4fee8f9edc8f1be6601214884ed6202b2a186ecef5b89dfabdf3f6a92bfff"}
Oct 30 13:17:07.294716 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:07.294685    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-t7zm7"
Oct 30 13:17:07.328622 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:07.328568    4943 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-w2nsg" podStartSLOduration=1.3285505450000001 podStartE2EDuration="1.328550545s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:17:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:17:07.324712195 +0000 UTC m=+1203.112715925" watchObservedRunningTime="2026-10-30 13:17:07.328550545 +0000 UTC m=+1203.116554274"
Oct 30 13:17:07.627659 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:07.627630    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-t7zm7"]
Oct 30 13:17:07.628793 ip-10-0-142-84 kubenswrapper[4943]: W1030 13:17:07.628767    4943 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0ee4232a_d7fe_4aa9_963c_4bac5daa86c3.slice/crio-37ccfbf1ae85542fe9d71b4279204f9e06c34e9e1af9f834a11005b3939d87be WatchSource:0}: Error finding container 37ccfbf1ae85542fe9d71b4279204f9e06c34e9e1af9f834a11005b3939d87be: Status 404 returned error can't find the container with id 37ccfbf1ae85542fe9d71b4279204f9e06c34e9e1af9f834a11005b3939d87be
Oct 30 13:17:07.631447 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:07.631415    4943 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 13:17:07.631516 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:07.631480    4943 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 13:17:07.631563 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:07.631514    4943 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 13:17:07.758520 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:07.758489    4943 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-w2nsg"
Oct 30 13:17:07.990291 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:07.987635    4943 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w1-05942d/distinct-2-w1-05942d-kserve-855c7f7c5f-pwt79"
Oct 30 13:17:08.003430 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:08.003399    4943 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w1-05942d/distinct-2-w1-05942d-kserve-855c7f7c5f-pwt79"
Oct 30 13:17:08.315389 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:08.314481    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-t7zm7" event={"ID":"0ee4232a-d7fe-4aa9-963c-4bac5daa86c3","Type":"ContainerStarted","Data":"b08612a8773ce4d0dae9d62e24dd8ab0b64f53425ac633e21d4c06ed876f2696"}
Oct 30 13:17:08.315389 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:08.314541    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-t7zm7" event={"ID":"0ee4232a-d7fe-4aa9-963c-4bac5daa86c3","Type":"ContainerStarted","Data":"37ccfbf1ae85542fe9d71b4279204f9e06c34e9e1af9f834a11005b3939d87be"}
Oct 30 13:17:08.327364 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:08.327336    4943 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-05942d/distinct-2-w1-05942d-kserve-855c7f7c5f-pwt79"
Oct 30 13:17:08.409290 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:08.405652    4943 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-t7zm7" podStartSLOduration=2.405628878 podStartE2EDuration="2.405628878s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:17:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:17:08.403668966 +0000 UTC m=+1204.191672697" watchObservedRunningTime="2026-10-30 13:17:08.405628878 +0000 UTC m=+1204.193632609"
Oct 30 13:17:08.760588 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:08.760553    4943 patch_prober.go:29] interesting pod/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-w2nsg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.96:15021/healthz/ready\": dial tcp 10.132.0.96:15021: connect: connection refused" start-of-body=
Oct 30 13:17:08.760769 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:08.760608    4943 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-w2nsg" podUID="558c9168-9f89-44f8-845b-74fe5345386c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.96:15021/healthz/ready\": dial tcp 10.132.0.96:15021: connect: connection refused"
Oct 30 13:17:09.307775 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:09.297622    4943 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-t7zm7"
Oct 30 13:17:09.309451 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:09.309414    4943 patch_prober.go:29] interesting pod/e2e-worker-w3-d0a18c-openshift-default-547d57db97-t7zm7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.97:15021/healthz/ready\": dial tcp 10.132.0.97:15021: connect: connection refused" start-of-body=
Oct 30 13:17:09.309688 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:09.309659    4943 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-t7zm7" podUID="0ee4232a-d7fe-4aa9-963c-4bac5daa86c3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.97:15021/healthz/ready\": dial tcp 10.132.0.97:15021: connect: connection refused"
Oct 30 13:17:09.758896 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:09.758861    4943 patch_prober.go:29] interesting pod/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-w2nsg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.96:15021/healthz/ready\": dial tcp 10.132.0.96:15021: connect: connection refused" start-of-body=
Oct 30 13:17:09.759050 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:09.758912    4943 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-w2nsg" podUID="558c9168-9f89-44f8-845b-74fe5345386c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.96:15021/healthz/ready\": dial tcp 10.132.0.96:15021: connect: connection refused"
Oct 30 13:17:10.295882 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:10.295837    4943 patch_prober.go:29] interesting pod/e2e-worker-w3-d0a18c-openshift-default-547d57db97-t7zm7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.97:15021/healthz/ready\": dial tcp 10.132.0.97:15021: connect: connection refused" start-of-body=
Oct 30 13:17:10.296207 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:10.296161    4943 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-t7zm7" podUID="0ee4232a-d7fe-4aa9-963c-4bac5daa86c3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.97:15021/healthz/ready\": dial tcp 10.132.0.97:15021: connect: connection refused"
Oct 30 13:17:10.759248 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:10.759205    4943 patch_prober.go:29] interesting pod/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-w2nsg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.96:15021/healthz/ready\": dial tcp 10.132.0.96:15021: connect: connection refused" start-of-body=
Oct 30 13:17:10.759755 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:10.759288    4943 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-w2nsg" podUID="558c9168-9f89-44f8-845b-74fe5345386c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.96:15021/healthz/ready\": dial tcp 10.132.0.96:15021: connect: connection refused"
Oct 30 13:17:11.298287 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:11.297747    4943 patch_prober.go:29] interesting pod/e2e-worker-w3-d0a18c-openshift-default-547d57db97-t7zm7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.97:15021/healthz/ready\": dial tcp 10.132.0.97:15021: connect: connection refused" start-of-body=
Oct 30 13:17:11.298287 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:11.297808    4943 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-t7zm7" podUID="0ee4232a-d7fe-4aa9-963c-4bac5daa86c3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.97:15021/healthz/ready\": dial tcp 10.132.0.97:15021: connect: connection refused"
Oct 30 13:17:11.758251 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:11.758222    4943 patch_prober.go:29] interesting pod/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-w2nsg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.96:15021/healthz/ready\": dial tcp 10.132.0.96:15021: connect: connection refused" start-of-body=
Oct 30 13:17:11.758369 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:11.758293    4943 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-w2nsg" podUID="558c9168-9f89-44f8-845b-74fe5345386c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.96:15021/healthz/ready\": dial tcp 10.132.0.96:15021: connect: connection refused"
Oct 30 13:17:12.295982 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:12.295941    4943 patch_prober.go:29] interesting pod/e2e-worker-w3-d0a18c-openshift-default-547d57db97-t7zm7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.97:15021/healthz/ready\": dial tcp 10.132.0.97:15021: connect: connection refused" start-of-body=
Oct 30 13:17:12.296556 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:12.296524    4943 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-t7zm7" podUID="0ee4232a-d7fe-4aa9-963c-4bac5daa86c3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.97:15021/healthz/ready\": dial tcp 10.132.0.97:15021: connect: connection refused"
Oct 30 13:17:12.533310 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:12.533248    4943 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-samples-operator_cluster-samples-operator-6ddb6c8b94-7dtcd_23297e74-48f9-47b7-bb5a-80189bdbf2c4/cluster-samples-operator/0.log"
Oct 30 13:17:12.533877 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:12.533848    4943 generic.go:356] "Generic (PLEG): container finished" podID="23297e74-48f9-47b7-bb5a-80189bdbf2c4" containerID="7528b304f0c5774afb2ac37234a9f817e8ef7234e2a705943e680328ae8b462f" exitCode=2
Oct 30 13:17:12.534011 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:12.533994    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-7dtcd" event={"ID":"23297e74-48f9-47b7-bb5a-80189bdbf2c4","Type":"ContainerDied","Data":"7528b304f0c5774afb2ac37234a9f817e8ef7234e2a705943e680328ae8b462f"}
Oct 30 13:17:12.535126 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:12.535103    4943 scope.go:122] "RemoveContainer" containerID="7528b304f0c5774afb2ac37234a9f817e8ef7234e2a705943e680328ae8b462f"
Oct 30 13:17:12.759118 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:12.758678    4943 patch_prober.go:29] interesting pod/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-w2nsg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.96:15021/healthz/ready\": dial tcp 10.132.0.96:15021: connect: connection refused" start-of-body=
Oct 30 13:17:12.759118 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:12.758743    4943 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-w2nsg" podUID="558c9168-9f89-44f8-845b-74fe5345386c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.96:15021/healthz/ready\": dial tcp 10.132.0.96:15021: connect: connection refused"
Oct 30 13:17:13.295653 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:13.295612    4943 patch_prober.go:29] interesting pod/e2e-worker-w3-d0a18c-openshift-default-547d57db97-t7zm7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.97:15021/healthz/ready\": dial tcp 10.132.0.97:15021: connect: connection refused" start-of-body=
Oct 30 13:17:13.296083 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:13.296049    4943 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-t7zm7" podUID="0ee4232a-d7fe-4aa9-963c-4bac5daa86c3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.97:15021/healthz/ready\": dial tcp 10.132.0.97:15021: connect: connection refused"
Oct 30 13:17:13.543715 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:13.543679    4943 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-samples-operator_cluster-samples-operator-6ddb6c8b94-7dtcd_23297e74-48f9-47b7-bb5a-80189bdbf2c4/cluster-samples-operator/0.log"
Oct 30 13:17:13.543891 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:13.543796    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-7dtcd" event={"ID":"23297e74-48f9-47b7-bb5a-80189bdbf2c4","Type":"ContainerStarted","Data":"ae3f6fdd5c37f48e6337dc7e97d3c37602a6eb22ff6f6296383b77375d08f806"}
Oct 30 13:17:13.761720 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:13.761680    4943 patch_prober.go:29] interesting pod/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-w2nsg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.96:15021/healthz/ready\": dial tcp 10.132.0.96:15021: connect: connection refused" start-of-body=
Oct 30 13:17:13.761921 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:13.761742    4943 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-w2nsg" podUID="558c9168-9f89-44f8-845b-74fe5345386c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.96:15021/healthz/ready\": dial tcp 10.132.0.96:15021: connect: connection refused"
Oct 30 13:17:14.295561 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:14.295514    4943 patch_prober.go:29] interesting pod/e2e-worker-w3-d0a18c-openshift-default-547d57db97-t7zm7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.97:15021/healthz/ready\": dial tcp 10.132.0.97:15021: connect: connection refused" start-of-body=
Oct 30 13:17:14.295727 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:14.295582    4943 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-t7zm7" podUID="0ee4232a-d7fe-4aa9-963c-4bac5daa86c3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.97:15021/healthz/ready\": dial tcp 10.132.0.97:15021: connect: connection refused"
Oct 30 13:17:14.763032 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:14.763002    4943 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-w2nsg"
Oct 30 13:17:14.763531 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:14.763511    4943 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-w2nsg"
Oct 30 13:17:14.764400 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:14.764374    4943 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-w2nsg"
Oct 30 13:17:15.398352 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:15.398232    4943 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-t7zm7"
Oct 30 13:17:15.398814 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:15.398788    4943 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-t7zm7"
Oct 30 13:17:15.399855 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:15.399822    4943 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-t7zm7"
Oct 30 13:17:21.270364 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:21.270325    4943 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-c98s9"]
Oct 30 13:17:21.304285 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:21.294552    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-c98s9"]
Oct 30 13:17:21.328286 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:21.304653    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-c98s9"
Oct 30 13:17:21.435287 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:21.384061    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/094f15a2-03fc-4322-9bcb-c190f5a6d8c4-workload-socket\") pod \"e2e-worker-w0-031737-openshift-default-64b8f46f47-c98s9\" (UID: \"094f15a2-03fc-4322-9bcb-c190f5a6d8c4\") " pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-c98s9"
Oct 30 13:17:21.435287 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:21.421920    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/094f15a2-03fc-4322-9bcb-c190f5a6d8c4-istio-envoy\") pod \"e2e-worker-w0-031737-openshift-default-64b8f46f47-c98s9\" (UID: \"094f15a2-03fc-4322-9bcb-c190f5a6d8c4\") " pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-c98s9"
Oct 30 13:17:21.435287 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:21.421968    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/094f15a2-03fc-4322-9bcb-c190f5a6d8c4-istio-token\") pod \"e2e-worker-w0-031737-openshift-default-64b8f46f47-c98s9\" (UID: \"094f15a2-03fc-4322-9bcb-c190f5a6d8c4\") " pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-c98s9"
Oct 30 13:17:21.435287 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:21.422007    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/094f15a2-03fc-4322-9bcb-c190f5a6d8c4-istiod-ca-cert\") pod \"e2e-worker-w0-031737-openshift-default-64b8f46f47-c98s9\" (UID: \"094f15a2-03fc-4322-9bcb-c190f5a6d8c4\") " pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-c98s9"
Oct 30 13:17:21.435287 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:21.422125    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/094f15a2-03fc-4322-9bcb-c190f5a6d8c4-credential-socket\") pod \"e2e-worker-w0-031737-openshift-default-64b8f46f47-c98s9\" (UID: \"094f15a2-03fc-4322-9bcb-c190f5a6d8c4\") " pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-c98s9"
Oct 30 13:17:21.435287 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:21.422235    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z65dz\" (UniqueName: \"kubernetes.io/projected/094f15a2-03fc-4322-9bcb-c190f5a6d8c4-kube-api-access-z65dz\") pod \"e2e-worker-w0-031737-openshift-default-64b8f46f47-c98s9\" (UID: \"094f15a2-03fc-4322-9bcb-c190f5a6d8c4\") " pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-c98s9"
Oct 30 13:17:21.435287 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:21.422327    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/094f15a2-03fc-4322-9bcb-c190f5a6d8c4-workload-certs\") pod \"e2e-worker-w0-031737-openshift-default-64b8f46f47-c98s9\" (UID: \"094f15a2-03fc-4322-9bcb-c190f5a6d8c4\") " pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-c98s9"
Oct 30 13:17:21.435287 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:21.422444    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/094f15a2-03fc-4322-9bcb-c190f5a6d8c4-istio-podinfo\") pod \"e2e-worker-w0-031737-openshift-default-64b8f46f47-c98s9\" (UID: \"094f15a2-03fc-4322-9bcb-c190f5a6d8c4\") " pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-c98s9"
Oct 30 13:17:21.435287 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:21.422489    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/094f15a2-03fc-4322-9bcb-c190f5a6d8c4-istio-data\") pod \"e2e-worker-w0-031737-openshift-default-64b8f46f47-c98s9\" (UID: \"094f15a2-03fc-4322-9bcb-c190f5a6d8c4\") " pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-c98s9"
Oct 30 13:17:21.552054 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:21.535069    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/094f15a2-03fc-4322-9bcb-c190f5a6d8c4-credential-socket\") pod \"e2e-worker-w0-031737-openshift-default-64b8f46f47-c98s9\" (UID: \"094f15a2-03fc-4322-9bcb-c190f5a6d8c4\") " pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-c98s9"
Oct 30 13:17:21.552054 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:21.535156    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z65dz\" (UniqueName: \"kubernetes.io/projected/094f15a2-03fc-4322-9bcb-c190f5a6d8c4-kube-api-access-z65dz\") pod \"e2e-worker-w0-031737-openshift-default-64b8f46f47-c98s9\" (UID: \"094f15a2-03fc-4322-9bcb-c190f5a6d8c4\") " pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-c98s9"
Oct 30 13:17:21.552054 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:21.535187    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/094f15a2-03fc-4322-9bcb-c190f5a6d8c4-workload-certs\") pod \"e2e-worker-w0-031737-openshift-default-64b8f46f47-c98s9\" (UID: \"094f15a2-03fc-4322-9bcb-c190f5a6d8c4\") " pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-c98s9"
Oct 30 13:17:21.552054 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:21.535254    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/094f15a2-03fc-4322-9bcb-c190f5a6d8c4-istio-podinfo\") pod \"e2e-worker-w0-031737-openshift-default-64b8f46f47-c98s9\" (UID: \"094f15a2-03fc-4322-9bcb-c190f5a6d8c4\") " pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-c98s9"
Oct 30 13:17:21.552054 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:21.535301    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/094f15a2-03fc-4322-9bcb-c190f5a6d8c4-istio-data\") pod \"e2e-worker-w0-031737-openshift-default-64b8f46f47-c98s9\" (UID: \"094f15a2-03fc-4322-9bcb-c190f5a6d8c4\") " pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-c98s9"
Oct 30 13:17:21.552054 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:21.535394    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/094f15a2-03fc-4322-9bcb-c190f5a6d8c4-workload-socket\") pod \"e2e-worker-w0-031737-openshift-default-64b8f46f47-c98s9\" (UID: \"094f15a2-03fc-4322-9bcb-c190f5a6d8c4\") " pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-c98s9"
Oct 30 13:17:21.552054 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:21.535422    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/094f15a2-03fc-4322-9bcb-c190f5a6d8c4-istio-envoy\") pod \"e2e-worker-w0-031737-openshift-default-64b8f46f47-c98s9\" (UID: \"094f15a2-03fc-4322-9bcb-c190f5a6d8c4\") " pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-c98s9"
Oct 30 13:17:21.552054 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:21.535450    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/094f15a2-03fc-4322-9bcb-c190f5a6d8c4-istio-token\") pod \"e2e-worker-w0-031737-openshift-default-64b8f46f47-c98s9\" (UID: \"094f15a2-03fc-4322-9bcb-c190f5a6d8c4\") " pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-c98s9"
Oct 30 13:17:21.552054 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:21.535471    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/094f15a2-03fc-4322-9bcb-c190f5a6d8c4-istiod-ca-cert\") pod \"e2e-worker-w0-031737-openshift-default-64b8f46f47-c98s9\" (UID: \"094f15a2-03fc-4322-9bcb-c190f5a6d8c4\") " pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-c98s9"
Oct 30 13:17:21.552054 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:21.536432    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/094f15a2-03fc-4322-9bcb-c190f5a6d8c4-istiod-ca-cert\") pod \"e2e-worker-w0-031737-openshift-default-64b8f46f47-c98s9\" (UID: \"094f15a2-03fc-4322-9bcb-c190f5a6d8c4\") " pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-c98s9"
Oct 30 13:17:21.552054 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:21.536772    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/094f15a2-03fc-4322-9bcb-c190f5a6d8c4-credential-socket\") pod \"e2e-worker-w0-031737-openshift-default-64b8f46f47-c98s9\" (UID: \"094f15a2-03fc-4322-9bcb-c190f5a6d8c4\") " pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-c98s9"
Oct 30 13:17:21.552054 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:21.537325    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/094f15a2-03fc-4322-9bcb-c190f5a6d8c4-workload-certs\") pod \"e2e-worker-w0-031737-openshift-default-64b8f46f47-c98s9\" (UID: \"094f15a2-03fc-4322-9bcb-c190f5a6d8c4\") " pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-c98s9"
Oct 30 13:17:21.565582 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:21.565541    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/094f15a2-03fc-4322-9bcb-c190f5a6d8c4-workload-socket\") pod \"e2e-worker-w0-031737-openshift-default-64b8f46f47-c98s9\" (UID: \"094f15a2-03fc-4322-9bcb-c190f5a6d8c4\") " pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-c98s9"
Oct 30 13:17:21.565897 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:21.565867    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/094f15a2-03fc-4322-9bcb-c190f5a6d8c4-istio-data\") pod \"e2e-worker-w0-031737-openshift-default-64b8f46f47-c98s9\" (UID: \"094f15a2-03fc-4322-9bcb-c190f5a6d8c4\") " pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-c98s9"
Oct 30 13:17:21.586923 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:21.586891    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/094f15a2-03fc-4322-9bcb-c190f5a6d8c4-istio-envoy\") pod \"e2e-worker-w0-031737-openshift-default-64b8f46f47-c98s9\" (UID: \"094f15a2-03fc-4322-9bcb-c190f5a6d8c4\") " pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-c98s9"
Oct 30 13:17:21.632285 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:21.631711    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z65dz\" (UniqueName: \"kubernetes.io/projected/094f15a2-03fc-4322-9bcb-c190f5a6d8c4-kube-api-access-z65dz\") pod \"e2e-worker-w0-031737-openshift-default-64b8f46f47-c98s9\" (UID: \"094f15a2-03fc-4322-9bcb-c190f5a6d8c4\") " pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-c98s9"
Oct 30 13:17:21.643284 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:21.632529    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/094f15a2-03fc-4322-9bcb-c190f5a6d8c4-istio-token\") pod \"e2e-worker-w0-031737-openshift-default-64b8f46f47-c98s9\" (UID: \"094f15a2-03fc-4322-9bcb-c190f5a6d8c4\") " pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-c98s9"
Oct 30 13:17:21.643284 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:21.634715    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/094f15a2-03fc-4322-9bcb-c190f5a6d8c4-istio-podinfo\") pod \"e2e-worker-w0-031737-openshift-default-64b8f46f47-c98s9\" (UID: \"094f15a2-03fc-4322-9bcb-c190f5a6d8c4\") " pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-c98s9"
Oct 30 13:17:21.662234 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:21.662198    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-c98s9"
Oct 30 13:17:22.099776 ip-10-0-142-84 kubenswrapper[4943]: W1030 13:17:22.099742    4943 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod094f15a2_03fc_4322_9bcb_c190f5a6d8c4.slice/crio-5c5ef704252d3e48acf5b80058a383df624a126e831e62b832660f35ca97f76b WatchSource:0}: Error finding container 5c5ef704252d3e48acf5b80058a383df624a126e831e62b832660f35ca97f76b: Status 404 returned error can't find the container with id 5c5ef704252d3e48acf5b80058a383df624a126e831e62b832660f35ca97f76b
Oct 30 13:17:22.101105 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:22.101078    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-c98s9"]
Oct 30 13:17:22.102302 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:22.102239    4943 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 13:17:22.102394 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:22.102336    4943 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 13:17:22.102394 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:22.102370    4943 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 13:17:22.781426 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:22.781383    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-c98s9" event={"ID":"094f15a2-03fc-4322-9bcb-c190f5a6d8c4","Type":"ContainerStarted","Data":"d97dcbe0e2d9b367fb9764d646726b47dc3956da80a55b1a9441ee67cbd98ca9"}
Oct 30 13:17:22.782044 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:22.782018    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-c98s9" event={"ID":"094f15a2-03fc-4322-9bcb-c190f5a6d8c4","Type":"ContainerStarted","Data":"5c5ef704252d3e48acf5b80058a383df624a126e831e62b832660f35ca97f76b"}
Oct 30 13:17:22.884285 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:22.883333    4943 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-c98s9" podStartSLOduration=1.883313552 podStartE2EDuration="1.883313552s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:17:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:17:22.859777304 +0000 UTC m=+1218.647781034" watchObservedRunningTime="2026-10-30 13:17:22.883313552 +0000 UTC m=+1218.671317281"
Oct 30 13:17:23.665700 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:23.665665    4943 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-c98s9"
Oct 30 13:17:23.687433 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:23.687392    4943 patch_prober.go:29] interesting pod/e2e-worker-w0-031737-openshift-default-64b8f46f47-c98s9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.98:15021/healthz/ready\": dial tcp 10.132.0.98:15021: connect: connection refused" start-of-body=
Oct 30 13:17:23.687616 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:23.687446    4943 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-c98s9" podUID="094f15a2-03fc-4322-9bcb-c190f5a6d8c4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.98:15021/healthz/ready\": dial tcp 10.132.0.98:15021: connect: connection refused"
Oct 30 13:17:24.686471 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:24.686430    4943 patch_prober.go:29] interesting pod/e2e-worker-w0-031737-openshift-default-64b8f46f47-c98s9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.98:15021/healthz/ready\": dial tcp 10.132.0.98:15021: connect: connection refused" start-of-body=
Oct 30 13:17:24.687005 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:24.686500    4943 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-c98s9" podUID="094f15a2-03fc-4322-9bcb-c190f5a6d8c4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.98:15021/healthz/ready\": dial tcp 10.132.0.98:15021: connect: connection refused"
Oct 30 13:17:25.683289 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:25.669672    4943 patch_prober.go:29] interesting pod/e2e-worker-w0-031737-openshift-default-64b8f46f47-c98s9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.98:15021/healthz/ready\": dial tcp 10.132.0.98:15021: connect: connection refused" start-of-body=
Oct 30 13:17:25.683289 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:25.669776    4943 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-c98s9" podUID="094f15a2-03fc-4322-9bcb-c190f5a6d8c4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.98:15021/healthz/ready\": dial tcp 10.132.0.98:15021: connect: connection refused"
Oct 30 13:17:26.685687 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:26.685649    4943 patch_prober.go:29] interesting pod/e2e-worker-w0-031737-openshift-default-64b8f46f47-c98s9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.98:15021/healthz/ready\": dial tcp 10.132.0.98:15021: connect: connection refused" start-of-body=
Oct 30 13:17:26.686354 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:26.686304    4943 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-c98s9" podUID="094f15a2-03fc-4322-9bcb-c190f5a6d8c4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.98:15021/healthz/ready\": dial tcp 10.132.0.98:15021: connect: connection refused"
Oct 30 13:17:27.669286 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:27.668355    4943 patch_prober.go:29] interesting pod/e2e-worker-w0-031737-openshift-default-64b8f46f47-c98s9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.98:15021/healthz/ready\": dial tcp 10.132.0.98:15021: connect: connection refused" start-of-body=
Oct 30 13:17:27.669286 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:27.668463    4943 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-c98s9" podUID="094f15a2-03fc-4322-9bcb-c190f5a6d8c4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.98:15021/healthz/ready\": dial tcp 10.132.0.98:15021: connect: connection refused"
Oct 30 13:17:28.920993 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:28.920943    4943 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-c98s9"
Oct 30 13:17:28.921976 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:28.921945    4943 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-c98s9"
Oct 30 13:17:28.922970 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:28.922942    4943 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-c98s9"
Oct 30 13:17:32.435889 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:32.435854    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-hdmrm"]
Oct 30 13:17:35.332286 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:35.323359    4943 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-05942d/distinct-2-w1-05942d-kserve-855c7f7c5f-pwt79" podUID="3dac65dc-7a67-4fdb-840a-765235aaa5bf" containerName="main" probeResult="failure" output="Get \"https://10.132.0.95:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:17:35.556291 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:35.552362    4943 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-ab3176/simulator-w6-ab3176-kserve-6d9b64549f-dc8xz" podUID="da306792-4f80-4df4-aa44-82991ab856e4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.69:8000/ready\": context deadline exceeded"
Oct 30 13:17:35.563387 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:35.563141    4943 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-d0a18c/distinct-w3-d0a18c-kserve-86479f6bbf-5zcqq" podUID="63dace4c-7832-47dd-84ab-2c91483a5563" containerName="main" probeResult="failure" output="Get \"https://10.132.0.89:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:17:35.716414 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:35.716359    4943 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w6-ab3176/simulator-w6-ab3176-kserve-6d9b64549f-dc8xz" podUID="da306792-4f80-4df4-aa44-82991ab856e4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.69:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:17:35.886432 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:35.886372    4943 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-f8a55e/distinct-w5-f8a55e-kserve-894fbd994-s2vsp" podUID="30f54ef4-28c2-4bf2-9feb-e2b0f1232c9f" containerName="main" probeResult="failure" output="Get \"https://10.132.0.92:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:17:36.047375 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:36.033410    4943 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-d0a18c/distinct-2-w3-d0a18c-kserve-85f8ff4846-rc44r" podUID="d65e47d2-4461-406a-9b71-bb3fac4262f0" containerName="main" probeResult="failure" output="Get \"https://10.132.0.93:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:17:36.248547 ip-10-0-142-84 kubenswrapper[4943]: E1030 13:17:36.248500    4943 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-knqz6" containerName="extractor"
Oct 30 13:17:42.425675 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:42.425634    4943 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-9789ebd6-tlflc"]
Oct 30 13:17:42.431594 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:42.431570    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-9789ebd6-tlflc"
Oct 30 13:17:42.463208 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:42.462134    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-9789ebd6-tlflc"]
Oct 30 13:17:42.464666 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:42.464634    4943 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 13:17:42.464782 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:42.464651    4943 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-p9z57\""
Oct 30 13:17:42.494415 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:42.494385    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/1860f3ec-4b40-4a11-9bd0-b26bcf796b90-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-9789ebd6-tlflc\" (UID: \"1860f3ec-4b40-4a11-9bd0-b26bcf796b90\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-9789ebd6-tlflc"
Oct 30 13:17:42.494588 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:42.494430    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wp72x\" (UniqueName: \"kubernetes.io/projected/1860f3ec-4b40-4a11-9bd0-b26bcf796b90-kube-api-access-wp72x\") pod \"maas-api-revoke-sub-9789ebd6-tlflc\" (UID: \"1860f3ec-4b40-4a11-9bd0-b26bcf796b90\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-9789ebd6-tlflc"
Oct 30 13:17:42.494588 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:42.494544    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/1860f3ec-4b40-4a11-9bd0-b26bcf796b90-maas-api-xks-ca\") pod \"maas-api-revoke-sub-9789ebd6-tlflc\" (UID: \"1860f3ec-4b40-4a11-9bd0-b26bcf796b90\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-9789ebd6-tlflc"
Oct 30 13:17:42.596036 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:42.596012    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/1860f3ec-4b40-4a11-9bd0-b26bcf796b90-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-9789ebd6-tlflc\" (UID: \"1860f3ec-4b40-4a11-9bd0-b26bcf796b90\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-9789ebd6-tlflc"
Oct 30 13:17:42.596167 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:42.596046    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wp72x\" (UniqueName: \"kubernetes.io/projected/1860f3ec-4b40-4a11-9bd0-b26bcf796b90-kube-api-access-wp72x\") pod \"maas-api-revoke-sub-9789ebd6-tlflc\" (UID: \"1860f3ec-4b40-4a11-9bd0-b26bcf796b90\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-9789ebd6-tlflc"
Oct 30 13:17:42.596167 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:42.596084    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/1860f3ec-4b40-4a11-9bd0-b26bcf796b90-maas-api-xks-ca\") pod \"maas-api-revoke-sub-9789ebd6-tlflc\" (UID: \"1860f3ec-4b40-4a11-9bd0-b26bcf796b90\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-9789ebd6-tlflc"
Oct 30 13:17:42.596795 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:42.596771    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/1860f3ec-4b40-4a11-9bd0-b26bcf796b90-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-9789ebd6-tlflc\" (UID: \"1860f3ec-4b40-4a11-9bd0-b26bcf796b90\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-9789ebd6-tlflc"
Oct 30 13:17:42.598496 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:42.598468    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/1860f3ec-4b40-4a11-9bd0-b26bcf796b90-maas-api-xks-ca\") pod \"maas-api-revoke-sub-9789ebd6-tlflc\" (UID: \"1860f3ec-4b40-4a11-9bd0-b26bcf796b90\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-9789ebd6-tlflc"
Oct 30 13:17:42.622360 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:42.622339    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wp72x\" (UniqueName: \"kubernetes.io/projected/1860f3ec-4b40-4a11-9bd0-b26bcf796b90-kube-api-access-wp72x\") pod \"maas-api-revoke-sub-9789ebd6-tlflc\" (UID: \"1860f3ec-4b40-4a11-9bd0-b26bcf796b90\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-9789ebd6-tlflc"
Oct 30 13:17:42.746091 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:42.746066    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-9789ebd6-tlflc"
Oct 30 13:17:43.064252 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:43.064227    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-9789ebd6-tlflc"]
Oct 30 13:17:43.066592 ip-10-0-142-84 kubenswrapper[4943]: W1030 13:17:43.066561    4943 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1860f3ec_4b40_4a11_9bd0_b26bcf796b90.slice/crio-f7dada9c9857fb890acf74c728926220e11689979b9935d77471ec014fc79461 WatchSource:0}: Error finding container f7dada9c9857fb890acf74c728926220e11689979b9935d77471ec014fc79461: Status 404 returned error can't find the container with id f7dada9c9857fb890acf74c728926220e11689979b9935d77471ec014fc79461
Oct 30 13:17:43.155226 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:43.155196    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-9789ebd6-tlflc" event={"ID":"1860f3ec-4b40-4a11-9bd0-b26bcf796b90","Type":"ContainerStarted","Data":"f7dada9c9857fb890acf74c728926220e11689979b9935d77471ec014fc79461"}
Oct 30 13:17:44.164408 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:44.164335    4943 generic.go:356] "Generic (PLEG): container finished" podID="1860f3ec-4b40-4a11-9bd0-b26bcf796b90" containerID="504f6f77769160d06b2691fc88b5fb1fb4be437ec6185b98f75df665abe250b4" exitCode=0
Oct 30 13:17:44.164408 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:44.164392    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-9789ebd6-tlflc" event={"ID":"1860f3ec-4b40-4a11-9bd0-b26bcf796b90","Type":"ContainerDied","Data":"504f6f77769160d06b2691fc88b5fb1fb4be437ec6185b98f75df665abe250b4"}
Oct 30 13:17:45.379919 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:45.379892    4943 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-9789ebd6-tlflc"
Oct 30 13:17:45.540068 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:45.540034    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1860f3ec-4b40-4a11-9bd0-b26bcf796b90-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/1860f3ec-4b40-4a11-9bd0-b26bcf796b90-maas-api-openshift-ca\") pod \"1860f3ec-4b40-4a11-9bd0-b26bcf796b90\" (UID: \"1860f3ec-4b40-4a11-9bd0-b26bcf796b90\") "
Oct 30 13:17:45.540277 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:45.540238    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/1860f3ec-4b40-4a11-9bd0-b26bcf796b90-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/1860f3ec-4b40-4a11-9bd0-b26bcf796b90-maas-api-xks-ca\") pod \"1860f3ec-4b40-4a11-9bd0-b26bcf796b90\" (UID: \"1860f3ec-4b40-4a11-9bd0-b26bcf796b90\") "
Oct 30 13:17:45.540364 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:45.540348    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1860f3ec-4b40-4a11-9bd0-b26bcf796b90-kube-api-access-wp72x\" (UniqueName: \"kubernetes.io/projected/1860f3ec-4b40-4a11-9bd0-b26bcf796b90-kube-api-access-wp72x\") pod \"1860f3ec-4b40-4a11-9bd0-b26bcf796b90\" (UID: \"1860f3ec-4b40-4a11-9bd0-b26bcf796b90\") "
Oct 30 13:17:45.540430 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:45.540394    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1860f3ec-4b40-4a11-9bd0-b26bcf796b90-maas-api-openshift-ca" pod "1860f3ec-4b40-4a11-9bd0-b26bcf796b90" (UID: "1860f3ec-4b40-4a11-9bd0-b26bcf796b90"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 13:17:45.540773 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:45.540753    4943 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/1860f3ec-4b40-4a11-9bd0-b26bcf796b90-maas-api-openshift-ca\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:17:45.542321 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:45.542286    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1860f3ec-4b40-4a11-9bd0-b26bcf796b90-kube-api-access-wp72x" pod "1860f3ec-4b40-4a11-9bd0-b26bcf796b90" (UID: "1860f3ec-4b40-4a11-9bd0-b26bcf796b90"). InnerVolumeSpecName "kube-api-access-wp72x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:17:45.542321 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:45.542304    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1860f3ec-4b40-4a11-9bd0-b26bcf796b90-maas-api-xks-ca" pod "1860f3ec-4b40-4a11-9bd0-b26bcf796b90" (UID: "1860f3ec-4b40-4a11-9bd0-b26bcf796b90"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 13:17:45.642415 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:45.642386    4943 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/1860f3ec-4b40-4a11-9bd0-b26bcf796b90-maas-api-xks-ca\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:17:45.642415 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:45.642413    4943 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wp72x\" (UniqueName: \"kubernetes.io/projected/1860f3ec-4b40-4a11-9bd0-b26bcf796b90-kube-api-access-wp72x\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:17:46.184414 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:46.184382    4943 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-9789ebd6-tlflc"
Oct 30 13:17:46.184414 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:46.184395    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-9789ebd6-tlflc" event={"ID":"1860f3ec-4b40-4a11-9bd0-b26bcf796b90","Type":"ContainerDied","Data":"f7dada9c9857fb890acf74c728926220e11689979b9935d77471ec014fc79461"}
Oct 30 13:17:46.184628 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:46.184445    4943 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f7dada9c9857fb890acf74c728926220e11689979b9935d77471ec014fc79461"
Oct 30 13:17:52.177530 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:52.177496    4943 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-sfbdb"]
Oct 30 13:17:52.178329 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:52.178307    4943 cpu_manager.go:414] "containerMap: removing container" podUID="1860f3ec-4b40-4a11-9bd0-b26bcf796b90" containerName="revoke-keys"
Oct 30 13:17:52.178329 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:52.178329    4943 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1860f3ec-4b40-4a11-9bd0-b26bcf796b90" containerName="revoke-keys"
Oct 30 13:17:52.178554 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:52.178539    4943 memory_manager.go:365] "RemoveStaleState removing state" podUID="1860f3ec-4b40-4a11-9bd0-b26bcf796b90" containerName="revoke-keys"
Oct 30 13:17:52.182094 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:52.182072    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-sfbdb"
Oct 30 13:17:52.220520 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:52.220487    4943 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-9ndzw"]
Oct 30 13:17:52.239480 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:52.239451    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-sfbdb"]
Oct 30 13:17:52.240201 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:52.240183    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-9ndzw"
Oct 30 13:17:52.296570 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:52.296532    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-9ndzw"]
Oct 30 13:17:52.334540 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:52.334503    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/70c62fc1-3452-47f0-8d1f-5495e535cd95-istio-podinfo\") pod \"e2e-worker-w3-d0a18c-openshift-default-547d57db97-9ndzw\" (UID: \"70c62fc1-3452-47f0-8d1f-5495e535cd95\") " pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-9ndzw"
Oct 30 13:17:52.334717 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:52.334557    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/70c62fc1-3452-47f0-8d1f-5495e535cd95-istiod-ca-cert\") pod \"e2e-worker-w3-d0a18c-openshift-default-547d57db97-9ndzw\" (UID: \"70c62fc1-3452-47f0-8d1f-5495e535cd95\") " pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-9ndzw"
Oct 30 13:17:52.334717 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:52.334580    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/70c62fc1-3452-47f0-8d1f-5495e535cd95-credential-socket\") pod \"e2e-worker-w3-d0a18c-openshift-default-547d57db97-9ndzw\" (UID: \"70c62fc1-3452-47f0-8d1f-5495e535cd95\") " pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-9ndzw"
Oct 30 13:17:52.334717 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:52.334685    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4a720f44-f6a3-48b5-8ba4-04b3214486eb-credential-socket\") pod \"e2e-worker-w3-d0a18c-openshift-default-547d57db97-sfbdb\" (UID: \"4a720f44-f6a3-48b5-8ba4-04b3214486eb\") " pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-sfbdb"
Oct 30 13:17:52.334830 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:52.334735    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4a720f44-f6a3-48b5-8ba4-04b3214486eb-istiod-ca-cert\") pod \"e2e-worker-w3-d0a18c-openshift-default-547d57db97-sfbdb\" (UID: \"4a720f44-f6a3-48b5-8ba4-04b3214486eb\") " pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-sfbdb"
Oct 30 13:17:52.334830 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:52.334802    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4a720f44-f6a3-48b5-8ba4-04b3214486eb-istio-envoy\") pod \"e2e-worker-w3-d0a18c-openshift-default-547d57db97-sfbdb\" (UID: \"4a720f44-f6a3-48b5-8ba4-04b3214486eb\") " pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-sfbdb"
Oct 30 13:17:52.334893 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:52.334861    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4a720f44-f6a3-48b5-8ba4-04b3214486eb-workload-certs\") pod \"e2e-worker-w3-d0a18c-openshift-default-547d57db97-sfbdb\" (UID: \"4a720f44-f6a3-48b5-8ba4-04b3214486eb\") " pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-sfbdb"
Oct 30 13:17:52.334950 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:52.334936    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xjzlf\" (UniqueName: \"kubernetes.io/projected/4a720f44-f6a3-48b5-8ba4-04b3214486eb-kube-api-access-xjzlf\") pod \"e2e-worker-w3-d0a18c-openshift-default-547d57db97-sfbdb\" (UID: \"4a720f44-f6a3-48b5-8ba4-04b3214486eb\") " pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-sfbdb"
Oct 30 13:17:52.334997 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:52.334961    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/70c62fc1-3452-47f0-8d1f-5495e535cd95-istio-data\") pod \"e2e-worker-w3-d0a18c-openshift-default-547d57db97-9ndzw\" (UID: \"70c62fc1-3452-47f0-8d1f-5495e535cd95\") " pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-9ndzw"
Oct 30 13:17:52.335089 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:52.335070    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/70c62fc1-3452-47f0-8d1f-5495e535cd95-istio-envoy\") pod \"e2e-worker-w3-d0a18c-openshift-default-547d57db97-9ndzw\" (UID: \"70c62fc1-3452-47f0-8d1f-5495e535cd95\") " pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-9ndzw"
Oct 30 13:17:52.335154 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:52.335110    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/70c62fc1-3452-47f0-8d1f-5495e535cd95-workload-socket\") pod \"e2e-worker-w3-d0a18c-openshift-default-547d57db97-9ndzw\" (UID: \"70c62fc1-3452-47f0-8d1f-5495e535cd95\") " pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-9ndzw"
Oct 30 13:17:52.335206 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:52.335193    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/70c62fc1-3452-47f0-8d1f-5495e535cd95-workload-certs\") pod \"e2e-worker-w3-d0a18c-openshift-default-547d57db97-9ndzw\" (UID: \"70c62fc1-3452-47f0-8d1f-5495e535cd95\") " pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-9ndzw"
Oct 30 13:17:52.335253 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:52.335213    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t87zw\" (UniqueName: \"kubernetes.io/projected/70c62fc1-3452-47f0-8d1f-5495e535cd95-kube-api-access-t87zw\") pod \"e2e-worker-w3-d0a18c-openshift-default-547d57db97-9ndzw\" (UID: \"70c62fc1-3452-47f0-8d1f-5495e535cd95\") " pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-9ndzw"
Oct 30 13:17:52.335253 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:52.335237    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4a720f44-f6a3-48b5-8ba4-04b3214486eb-workload-socket\") pod \"e2e-worker-w3-d0a18c-openshift-default-547d57db97-sfbdb\" (UID: \"4a720f44-f6a3-48b5-8ba4-04b3214486eb\") " pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-sfbdb"
Oct 30 13:17:52.335253 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:52.335253    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/70c62fc1-3452-47f0-8d1f-5495e535cd95-istio-token\") pod \"e2e-worker-w3-d0a18c-openshift-default-547d57db97-9ndzw\" (UID: \"70c62fc1-3452-47f0-8d1f-5495e535cd95\") " pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-9ndzw"
Oct 30 13:17:52.335380 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:52.335296    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4a720f44-f6a3-48b5-8ba4-04b3214486eb-istio-token\") pod \"e2e-worker-w3-d0a18c-openshift-default-547d57db97-sfbdb\" (UID: \"4a720f44-f6a3-48b5-8ba4-04b3214486eb\") " pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-sfbdb"
Oct 30 13:17:52.335380 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:52.335354    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4a720f44-f6a3-48b5-8ba4-04b3214486eb-istio-data\") pod \"e2e-worker-w3-d0a18c-openshift-default-547d57db97-sfbdb\" (UID: \"4a720f44-f6a3-48b5-8ba4-04b3214486eb\") " pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-sfbdb"
Oct 30 13:17:52.335380 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:52.335371    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4a720f44-f6a3-48b5-8ba4-04b3214486eb-istio-podinfo\") pod \"e2e-worker-w3-d0a18c-openshift-default-547d57db97-sfbdb\" (UID: \"4a720f44-f6a3-48b5-8ba4-04b3214486eb\") " pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-sfbdb"
Oct 30 13:17:52.437137 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:52.437054    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4a720f44-f6a3-48b5-8ba4-04b3214486eb-istio-data\") pod \"e2e-worker-w3-d0a18c-openshift-default-547d57db97-sfbdb\" (UID: \"4a720f44-f6a3-48b5-8ba4-04b3214486eb\") " pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-sfbdb"
Oct 30 13:17:52.437137 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:52.437096    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4a720f44-f6a3-48b5-8ba4-04b3214486eb-istio-podinfo\") pod \"e2e-worker-w3-d0a18c-openshift-default-547d57db97-sfbdb\" (UID: \"4a720f44-f6a3-48b5-8ba4-04b3214486eb\") " pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-sfbdb"
Oct 30 13:17:52.437137 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:52.437119    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/70c62fc1-3452-47f0-8d1f-5495e535cd95-istio-podinfo\") pod \"e2e-worker-w3-d0a18c-openshift-default-547d57db97-9ndzw\" (UID: \"70c62fc1-3452-47f0-8d1f-5495e535cd95\") " pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-9ndzw"
Oct 30 13:17:52.437540 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:52.437149    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/70c62fc1-3452-47f0-8d1f-5495e535cd95-istiod-ca-cert\") pod \"e2e-worker-w3-d0a18c-openshift-default-547d57db97-9ndzw\" (UID: \"70c62fc1-3452-47f0-8d1f-5495e535cd95\") " pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-9ndzw"
Oct 30 13:17:52.437540 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:52.437178    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/70c62fc1-3452-47f0-8d1f-5495e535cd95-credential-socket\") pod \"e2e-worker-w3-d0a18c-openshift-default-547d57db97-9ndzw\" (UID: \"70c62fc1-3452-47f0-8d1f-5495e535cd95\") " pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-9ndzw"
Oct 30 13:17:52.437540 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:52.437212    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4a720f44-f6a3-48b5-8ba4-04b3214486eb-credential-socket\") pod \"e2e-worker-w3-d0a18c-openshift-default-547d57db97-sfbdb\" (UID: \"4a720f44-f6a3-48b5-8ba4-04b3214486eb\") " pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-sfbdb"
Oct 30 13:17:52.437540 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:52.437349    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4a720f44-f6a3-48b5-8ba4-04b3214486eb-istiod-ca-cert\") pod \"e2e-worker-w3-d0a18c-openshift-default-547d57db97-sfbdb\" (UID: \"4a720f44-f6a3-48b5-8ba4-04b3214486eb\") " pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-sfbdb"
Oct 30 13:17:52.437540 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:52.437404    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4a720f44-f6a3-48b5-8ba4-04b3214486eb-istio-envoy\") pod \"e2e-worker-w3-d0a18c-openshift-default-547d57db97-sfbdb\" (UID: \"4a720f44-f6a3-48b5-8ba4-04b3214486eb\") " pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-sfbdb"
Oct 30 13:17:52.437815 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:52.437551    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4a720f44-f6a3-48b5-8ba4-04b3214486eb-workload-certs\") pod \"e2e-worker-w3-d0a18c-openshift-default-547d57db97-sfbdb\" (UID: \"4a720f44-f6a3-48b5-8ba4-04b3214486eb\") " pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-sfbdb"
Oct 30 13:17:52.437815 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:52.437599    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xjzlf\" (UniqueName: \"kubernetes.io/projected/4a720f44-f6a3-48b5-8ba4-04b3214486eb-kube-api-access-xjzlf\") pod \"e2e-worker-w3-d0a18c-openshift-default-547d57db97-sfbdb\" (UID: \"4a720f44-f6a3-48b5-8ba4-04b3214486eb\") " pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-sfbdb"
Oct 30 13:17:52.437815 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:52.437619    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4a720f44-f6a3-48b5-8ba4-04b3214486eb-istio-data\") pod \"e2e-worker-w3-d0a18c-openshift-default-547d57db97-sfbdb\" (UID: \"4a720f44-f6a3-48b5-8ba4-04b3214486eb\") " pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-sfbdb"
Oct 30 13:17:52.437815 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:52.437631    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/70c62fc1-3452-47f0-8d1f-5495e535cd95-istio-data\") pod \"e2e-worker-w3-d0a18c-openshift-default-547d57db97-9ndzw\" (UID: \"70c62fc1-3452-47f0-8d1f-5495e535cd95\") " pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-9ndzw"
Oct 30 13:17:52.437815 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:52.437679    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4a720f44-f6a3-48b5-8ba4-04b3214486eb-credential-socket\") pod \"e2e-worker-w3-d0a18c-openshift-default-547d57db97-sfbdb\" (UID: \"4a720f44-f6a3-48b5-8ba4-04b3214486eb\") " pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-sfbdb"
Oct 30 13:17:52.437815 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:52.437730    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/70c62fc1-3452-47f0-8d1f-5495e535cd95-credential-socket\") pod \"e2e-worker-w3-d0a18c-openshift-default-547d57db97-9ndzw\" (UID: \"70c62fc1-3452-47f0-8d1f-5495e535cd95\") " pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-9ndzw"
Oct 30 13:17:52.438134 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:52.437966    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/70c62fc1-3452-47f0-8d1f-5495e535cd95-istio-data\") pod \"e2e-worker-w3-d0a18c-openshift-default-547d57db97-9ndzw\" (UID: \"70c62fc1-3452-47f0-8d1f-5495e535cd95\") " pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-9ndzw"
Oct 30 13:17:52.438134 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:52.437970    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4a720f44-f6a3-48b5-8ba4-04b3214486eb-workload-certs\") pod \"e2e-worker-w3-d0a18c-openshift-default-547d57db97-sfbdb\" (UID: \"4a720f44-f6a3-48b5-8ba4-04b3214486eb\") " pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-sfbdb"
Oct 30 13:17:52.438134 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:52.438034    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/70c62fc1-3452-47f0-8d1f-5495e535cd95-istio-envoy\") pod \"e2e-worker-w3-d0a18c-openshift-default-547d57db97-9ndzw\" (UID: \"70c62fc1-3452-47f0-8d1f-5495e535cd95\") " pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-9ndzw"
Oct 30 13:17:52.438134 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:52.438074    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/70c62fc1-3452-47f0-8d1f-5495e535cd95-workload-socket\") pod \"e2e-worker-w3-d0a18c-openshift-default-547d57db97-9ndzw\" (UID: \"70c62fc1-3452-47f0-8d1f-5495e535cd95\") " pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-9ndzw"
Oct 30 13:17:52.438134 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:52.438103    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/70c62fc1-3452-47f0-8d1f-5495e535cd95-istiod-ca-cert\") pod \"e2e-worker-w3-d0a18c-openshift-default-547d57db97-9ndzw\" (UID: \"70c62fc1-3452-47f0-8d1f-5495e535cd95\") " pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-9ndzw"
Oct 30 13:17:52.438134 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:52.438131    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/70c62fc1-3452-47f0-8d1f-5495e535cd95-workload-certs\") pod \"e2e-worker-w3-d0a18c-openshift-default-547d57db97-9ndzw\" (UID: \"70c62fc1-3452-47f0-8d1f-5495e535cd95\") " pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-9ndzw"
Oct 30 13:17:52.438466 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:52.438159    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t87zw\" (UniqueName: \"kubernetes.io/projected/70c62fc1-3452-47f0-8d1f-5495e535cd95-kube-api-access-t87zw\") pod \"e2e-worker-w3-d0a18c-openshift-default-547d57db97-9ndzw\" (UID: \"70c62fc1-3452-47f0-8d1f-5495e535cd95\") " pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-9ndzw"
Oct 30 13:17:52.438466 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:52.438196    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4a720f44-f6a3-48b5-8ba4-04b3214486eb-workload-socket\") pod \"e2e-worker-w3-d0a18c-openshift-default-547d57db97-sfbdb\" (UID: \"4a720f44-f6a3-48b5-8ba4-04b3214486eb\") " pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-sfbdb"
Oct 30 13:17:52.438466 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:52.438221    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/70c62fc1-3452-47f0-8d1f-5495e535cd95-istio-token\") pod \"e2e-worker-w3-d0a18c-openshift-default-547d57db97-9ndzw\" (UID: \"70c62fc1-3452-47f0-8d1f-5495e535cd95\") " pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-9ndzw"
Oct 30 13:17:52.438466 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:52.438249    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4a720f44-f6a3-48b5-8ba4-04b3214486eb-istio-token\") pod \"e2e-worker-w3-d0a18c-openshift-default-547d57db97-sfbdb\" (UID: \"4a720f44-f6a3-48b5-8ba4-04b3214486eb\") " pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-sfbdb"
Oct 30 13:17:52.438466 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:52.438403    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4a720f44-f6a3-48b5-8ba4-04b3214486eb-istiod-ca-cert\") pod \"e2e-worker-w3-d0a18c-openshift-default-547d57db97-sfbdb\" (UID: \"4a720f44-f6a3-48b5-8ba4-04b3214486eb\") " pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-sfbdb"
Oct 30 13:17:52.438742 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:52.438509    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/70c62fc1-3452-47f0-8d1f-5495e535cd95-workload-socket\") pod \"e2e-worker-w3-d0a18c-openshift-default-547d57db97-9ndzw\" (UID: \"70c62fc1-3452-47f0-8d1f-5495e535cd95\") " pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-9ndzw"
Oct 30 13:17:52.438742 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:52.438632    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4a720f44-f6a3-48b5-8ba4-04b3214486eb-workload-socket\") pod \"e2e-worker-w3-d0a18c-openshift-default-547d57db97-sfbdb\" (UID: \"4a720f44-f6a3-48b5-8ba4-04b3214486eb\") " pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-sfbdb"
Oct 30 13:17:52.438958 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:52.438934    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/70c62fc1-3452-47f0-8d1f-5495e535cd95-workload-certs\") pod \"e2e-worker-w3-d0a18c-openshift-default-547d57db97-9ndzw\" (UID: \"70c62fc1-3452-47f0-8d1f-5495e535cd95\") " pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-9ndzw"
Oct 30 13:17:52.440889 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:52.440853    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/70c62fc1-3452-47f0-8d1f-5495e535cd95-istio-envoy\") pod \"e2e-worker-w3-d0a18c-openshift-default-547d57db97-9ndzw\" (UID: \"70c62fc1-3452-47f0-8d1f-5495e535cd95\") " pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-9ndzw"
Oct 30 13:17:52.440889 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:52.440879    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4a720f44-f6a3-48b5-8ba4-04b3214486eb-istio-envoy\") pod \"e2e-worker-w3-d0a18c-openshift-default-547d57db97-sfbdb\" (UID: \"4a720f44-f6a3-48b5-8ba4-04b3214486eb\") " pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-sfbdb"
Oct 30 13:17:52.441017 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:52.440943    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/70c62fc1-3452-47f0-8d1f-5495e535cd95-istio-podinfo\") pod \"e2e-worker-w3-d0a18c-openshift-default-547d57db97-9ndzw\" (UID: \"70c62fc1-3452-47f0-8d1f-5495e535cd95\") " pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-9ndzw"
Oct 30 13:17:52.441017 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:52.440943    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4a720f44-f6a3-48b5-8ba4-04b3214486eb-istio-podinfo\") pod \"e2e-worker-w3-d0a18c-openshift-default-547d57db97-sfbdb\" (UID: \"4a720f44-f6a3-48b5-8ba4-04b3214486eb\") " pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-sfbdb"
Oct 30 13:17:52.468776 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:52.468750    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xjzlf\" (UniqueName: \"kubernetes.io/projected/4a720f44-f6a3-48b5-8ba4-04b3214486eb-kube-api-access-xjzlf\") pod \"e2e-worker-w3-d0a18c-openshift-default-547d57db97-sfbdb\" (UID: \"4a720f44-f6a3-48b5-8ba4-04b3214486eb\") " pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-sfbdb"
Oct 30 13:17:52.473758 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:52.473734    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/70c62fc1-3452-47f0-8d1f-5495e535cd95-istio-token\") pod \"e2e-worker-w3-d0a18c-openshift-default-547d57db97-9ndzw\" (UID: \"70c62fc1-3452-47f0-8d1f-5495e535cd95\") " pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-9ndzw"
Oct 30 13:17:52.477046 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:52.477025    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4a720f44-f6a3-48b5-8ba4-04b3214486eb-istio-token\") pod \"e2e-worker-w3-d0a18c-openshift-default-547d57db97-sfbdb\" (UID: \"4a720f44-f6a3-48b5-8ba4-04b3214486eb\") " pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-sfbdb"
Oct 30 13:17:52.477574 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:52.477556    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t87zw\" (UniqueName: \"kubernetes.io/projected/70c62fc1-3452-47f0-8d1f-5495e535cd95-kube-api-access-t87zw\") pod \"e2e-worker-w3-d0a18c-openshift-default-547d57db97-9ndzw\" (UID: \"70c62fc1-3452-47f0-8d1f-5495e535cd95\") " pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-9ndzw"
Oct 30 13:17:52.499971 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:52.499951    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-sfbdb"
Oct 30 13:17:52.556325 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:52.556247    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-9ndzw"
Oct 30 13:17:52.940804 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:52.917673    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-sfbdb"]
Oct 30 13:17:52.982497 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:52.978037    4943 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 13:17:52.982497 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:52.978137    4943 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 13:17:52.982497 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:52.978185    4943 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 13:17:53.092469 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:53.087281    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-9ndzw"]
Oct 30 13:17:53.114326 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:53.108933    4943 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 13:17:53.114326 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:53.109031    4943 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 13:17:53.114326 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:53.109076    4943 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 13:17:53.395036 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:53.394987    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-9ndzw" event={"ID":"70c62fc1-3452-47f0-8d1f-5495e535cd95","Type":"ContainerStarted","Data":"7edaf6646307190bf2f10077a55ea9a16dbf84b8c86d56980ad39acd7a082e1a"}
Oct 30 13:17:53.397307 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:53.397277    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-sfbdb" event={"ID":"4a720f44-f6a3-48b5-8ba4-04b3214486eb","Type":"ContainerStarted","Data":"31b96717eacfe5e1fe916de9824de1d0dcef71d9f14dbbc8d6058318f04d8cd1"}
Oct 30 13:17:54.560790 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:54.560733    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-9ndzw" event={"ID":"70c62fc1-3452-47f0-8d1f-5495e535cd95","Type":"ContainerStarted","Data":"eead86c384c8ed3785b2c45cc493e7003a79c6c27a92ccc315f6cbca01aa5676"}
Oct 30 13:17:54.562396 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:54.562359    4943 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-9ndzw"
Oct 30 13:17:54.588774 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:54.588738    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-sfbdb" event={"ID":"4a720f44-f6a3-48b5-8ba4-04b3214486eb","Type":"ContainerStarted","Data":"a154f39e24a8c3c65170b5213a6d298fbfbe40344cc4164fa7794afbf59fc265"}
Oct 30 13:17:54.589039 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:54.563419    4943 patch_prober.go:29] interesting pod/e2e-worker-w3-d0a18c-openshift-default-547d57db97-9ndzw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.101:15021/healthz/ready\": dial tcp 10.132.0.101:15021: connect: connection refused" start-of-body=
Oct 30 13:17:54.589188 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:54.589164    4943 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-9ndzw" podUID="70c62fc1-3452-47f0-8d1f-5495e535cd95" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.101:15021/healthz/ready\": dial tcp 10.132.0.101:15021: connect: connection refused"
Oct 30 13:17:54.700286 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:54.690752    4943 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-9ndzw" podStartSLOduration=2.690730705 podStartE2EDuration="2.690730705s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:17:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:17:54.636786237 +0000 UTC m=+1250.424789981" watchObservedRunningTime="2026-10-30 13:17:54.690730705 +0000 UTC m=+1250.478734436"
Oct 30 13:17:54.754284 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:54.734007    4943 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-sfbdb" podStartSLOduration=2.733983048 podStartE2EDuration="2.733983048s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:17:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:17:54.703791777 +0000 UTC m=+1250.491795511" watchObservedRunningTime="2026-10-30 13:17:54.733983048 +0000 UTC m=+1250.521986780"
Oct 30 13:17:55.373286 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:55.369393    4943 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-05942d/distinct-2-w1-05942d-kserve-855c7f7c5f-pwt79" podUID="3dac65dc-7a67-4fdb-840a-765235aaa5bf" containerName="main" probeResult="failure" output="Get \"https://10.132.0.95:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:17:55.503920 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:55.501431    4943 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-sfbdb"
Oct 30 13:17:55.504327 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:55.504300    4943 patch_prober.go:29] interesting pod/e2e-worker-w3-d0a18c-openshift-default-547d57db97-sfbdb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.100:15021/healthz/ready\": dial tcp 10.132.0.100:15021: connect: connection refused" start-of-body=
Oct 30 13:17:55.504574 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:55.504530    4943 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-sfbdb" podUID="4a720f44-f6a3-48b5-8ba4-04b3214486eb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.100:15021/healthz/ready\": dial tcp 10.132.0.100:15021: connect: connection refused"
Oct 30 13:17:55.557905 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:55.557747    4943 patch_prober.go:29] interesting pod/e2e-worker-w3-d0a18c-openshift-default-547d57db97-9ndzw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.101:15021/healthz/ready\": dial tcp 10.132.0.101:15021: connect: connection refused" start-of-body=
Oct 30 13:17:55.558809 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:55.558188    4943 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-9ndzw" podUID="70c62fc1-3452-47f0-8d1f-5495e535cd95" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.101:15021/healthz/ready\": dial tcp 10.132.0.101:15021: connect: connection refused"
Oct 30 13:17:55.563344 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:55.563308    4943 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-d0a18c/distinct-w3-d0a18c-kserve-86479f6bbf-5zcqq" podUID="63dace4c-7832-47dd-84ab-2c91483a5563" containerName="main" probeResult="failure" output="Get \"https://10.132.0.89:8000/ready\": context deadline exceeded"
Oct 30 13:17:55.564614 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:55.564585    4943 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-ab3176/simulator-w6-ab3176-kserve-6d9b64549f-dc8xz" podUID="da306792-4f80-4df4-aa44-82991ab856e4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.69:8000/ready\": context deadline exceeded"
Oct 30 13:17:55.671671 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:55.671570    4943 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-031737/premium-w0-031737-kserve-58d5857cdc-vfnz9" podUID="79b1a638-b019-4c89-b850-01699ecac9ba" containerName="main" probeResult="failure" output="Get \"https://10.132.0.70:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:17:55.711304 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:55.710434    4943 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w6-ab3176/simulator-w6-ab3176-kserve-6d9b64549f-dc8xz" podUID="da306792-4f80-4df4-aa44-82991ab856e4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.69:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:17:55.804775 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:55.804725    4943 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-05942d/distinct-w1-05942d-kserve-5fb44dc664-mcbkc" podUID="d08eb359-cf9e-422c-a3a7-35fd173ab7b6" containerName="main" probeResult="failure" output="Get \"https://10.132.0.91:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:17:55.882553 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:55.882505    4943 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-f8a55e/distinct-w5-f8a55e-kserve-894fbd994-s2vsp" podUID="30f54ef4-28c2-4bf2-9feb-e2b0f1232c9f" containerName="main" probeResult="failure" output="Get \"https://10.132.0.92:8000/ready\": context deadline exceeded"
Oct 30 13:17:56.028452 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:56.028401    4943 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-d0a18c/distinct-2-w3-d0a18c-kserve-85f8ff4846-rc44r" podUID="d65e47d2-4461-406a-9b71-bb3fac4262f0" containerName="main" probeResult="failure" output="Get \"https://10.132.0.93:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:17:56.427342 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:56.370645    4943 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-05942d/distinct-2-w1-05942d-kserve-855c7f7c5f-pwt79" podUID="3dac65dc-7a67-4fdb-840a-765235aaa5bf" containerName="main" probeResult="failure" output="Get \"https://10.132.0.95:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:17:56.428378 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:56.428350    4943 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-05942d/distinct-2-w1-05942d-kserve-855c7f7c5f-pwt79"
Oct 30 13:17:56.502302 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:56.502245    4943 patch_prober.go:29] interesting pod/e2e-worker-w3-d0a18c-openshift-default-547d57db97-sfbdb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.100:15021/healthz/ready\": dial tcp 10.132.0.100:15021: connect: connection refused" start-of-body=
Oct 30 13:17:56.502568 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:56.502541    4943 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-sfbdb" podUID="4a720f44-f6a3-48b5-8ba4-04b3214486eb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.100:15021/healthz/ready\": dial tcp 10.132.0.100:15021: connect: connection refused"
Oct 30 13:17:56.560283 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:56.559968    4943 patch_prober.go:29] interesting pod/e2e-worker-w3-d0a18c-openshift-default-547d57db97-9ndzw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.101:15021/healthz/ready\": dial tcp 10.132.0.101:15021: connect: connection refused" start-of-body=
Oct 30 13:17:56.560283 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:56.560026    4943 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-9ndzw" podUID="70c62fc1-3452-47f0-8d1f-5495e535cd95" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.101:15021/healthz/ready\": dial tcp 10.132.0.101:15021: connect: connection refused"
Oct 30 13:17:56.760367 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:56.759390    4943 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-05942d/distinct-2-w1-05942d-kserve-855c7f7c5f-pwt79"
Oct 30 13:17:57.505963 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:57.505924    4943 patch_prober.go:29] interesting pod/e2e-worker-w3-d0a18c-openshift-default-547d57db97-sfbdb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.100:15021/healthz/ready\": dial tcp 10.132.0.100:15021: connect: connection refused" start-of-body=
Oct 30 13:17:57.506248 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:57.506211    4943 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-sfbdb" podUID="4a720f44-f6a3-48b5-8ba4-04b3214486eb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.100:15021/healthz/ready\": dial tcp 10.132.0.100:15021: connect: connection refused"
Oct 30 13:17:57.573093 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:57.572626    4943 patch_prober.go:29] interesting pod/e2e-worker-w3-d0a18c-openshift-default-547d57db97-9ndzw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.101:15021/healthz/ready\": dial tcp 10.132.0.101:15021: connect: connection refused" start-of-body=
Oct 30 13:17:57.573093 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:57.572687    4943 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-9ndzw" podUID="70c62fc1-3452-47f0-8d1f-5495e535cd95" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.101:15021/healthz/ready\": dial tcp 10.132.0.101:15021: connect: connection refused"
Oct 30 13:17:58.500682 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:58.500642    4943 patch_prober.go:29] interesting pod/e2e-worker-w3-d0a18c-openshift-default-547d57db97-sfbdb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.100:15021/healthz/ready\": dial tcp 10.132.0.100:15021: connect: connection refused" start-of-body=
Oct 30 13:17:58.501338 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:58.501305    4943 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-sfbdb" podUID="4a720f44-f6a3-48b5-8ba4-04b3214486eb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.100:15021/healthz/ready\": dial tcp 10.132.0.100:15021: connect: connection refused"
Oct 30 13:17:58.556927 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:58.556897    4943 patch_prober.go:29] interesting pod/e2e-worker-w3-d0a18c-openshift-default-547d57db97-9ndzw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.101:15021/healthz/ready\": dial tcp 10.132.0.101:15021: connect: connection refused" start-of-body=
Oct 30 13:17:58.557099 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:58.556946    4943 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-9ndzw" podUID="70c62fc1-3452-47f0-8d1f-5495e535cd95" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.101:15021/healthz/ready\": dial tcp 10.132.0.101:15021: connect: connection refused"
Oct 30 13:17:59.504285 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:59.504182    4943 patch_prober.go:29] interesting pod/e2e-worker-w3-d0a18c-openshift-default-547d57db97-sfbdb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.100:15021/healthz/ready\": dial tcp 10.132.0.100:15021: connect: connection refused" start-of-body=
Oct 30 13:17:59.504285 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:59.504243    4943 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-sfbdb" podUID="4a720f44-f6a3-48b5-8ba4-04b3214486eb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.100:15021/healthz/ready\": dial tcp 10.132.0.100:15021: connect: connection refused"
Oct 30 13:17:59.556651 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:59.556619    4943 patch_prober.go:29] interesting pod/e2e-worker-w3-d0a18c-openshift-default-547d57db97-9ndzw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.101:15021/healthz/ready\": dial tcp 10.132.0.101:15021: connect: connection refused" start-of-body=
Oct 30 13:17:59.556812 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:17:59.556667    4943 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-9ndzw" podUID="70c62fc1-3452-47f0-8d1f-5495e535cd95" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.101:15021/healthz/ready\": dial tcp 10.132.0.101:15021: connect: connection refused"
Oct 30 13:18:00.506285 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:00.505635    4943 patch_prober.go:29] interesting pod/e2e-worker-w3-d0a18c-openshift-default-547d57db97-sfbdb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.100:15021/healthz/ready\": dial tcp 10.132.0.100:15021: connect: connection refused" start-of-body=
Oct 30 13:18:00.506285 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:00.505701    4943 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-sfbdb" podUID="4a720f44-f6a3-48b5-8ba4-04b3214486eb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.100:15021/healthz/ready\": dial tcp 10.132.0.100:15021: connect: connection refused"
Oct 30 13:18:00.556605 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:00.556568    4943 patch_prober.go:29] interesting pod/e2e-worker-w3-d0a18c-openshift-default-547d57db97-9ndzw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.101:15021/healthz/ready\": dial tcp 10.132.0.101:15021: connect: connection refused" start-of-body=
Oct 30 13:18:00.556765 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:00.556628    4943 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-9ndzw" podUID="70c62fc1-3452-47f0-8d1f-5495e535cd95" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.101:15021/healthz/ready\": dial tcp 10.132.0.101:15021: connect: connection refused"
Oct 30 13:18:01.500620 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:01.500583    4943 patch_prober.go:29] interesting pod/e2e-worker-w3-d0a18c-openshift-default-547d57db97-sfbdb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.100:15021/healthz/ready\": dial tcp 10.132.0.100:15021: connect: connection refused" start-of-body=
Oct 30 13:18:01.500802 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:01.500648    4943 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-sfbdb" podUID="4a720f44-f6a3-48b5-8ba4-04b3214486eb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.100:15021/healthz/ready\": dial tcp 10.132.0.100:15021: connect: connection refused"
Oct 30 13:18:01.556991 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:01.556956    4943 patch_prober.go:29] interesting pod/e2e-worker-w3-d0a18c-openshift-default-547d57db97-9ndzw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.101:15021/healthz/ready\": dial tcp 10.132.0.101:15021: connect: connection refused" start-of-body=
Oct 30 13:18:01.557386 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:01.557017    4943 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-9ndzw" podUID="70c62fc1-3452-47f0-8d1f-5495e535cd95" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.101:15021/healthz/ready\": dial tcp 10.132.0.101:15021: connect: connection refused"
Oct 30 13:18:02.879157 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:02.879125    4943 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-9ndzw"
Oct 30 13:18:02.880386 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:02.880364    4943 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-9ndzw"
Oct 30 13:18:02.880962 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:02.880942    4943 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-9ndzw"
Oct 30 13:18:03.023772 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:03.023736    4943 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-sfbdb"
Oct 30 13:18:03.025579 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:03.024854    4943 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-sfbdb"
Oct 30 13:18:03.034480 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:03.034442    4943 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-sfbdb"
Oct 30 13:18:05.018921 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:05.018891    4943 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-76f0e08b-t78dl"]
Oct 30 13:18:05.024811 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:05.024792    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-76f0e08b-t78dl"
Oct 30 13:18:05.044383 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:05.044359    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-76f0e08b-t78dl"]
Oct 30 13:18:05.052249 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:05.052219    4943 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 13:18:05.052473 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:05.052449    4943 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-p9z57\""
Oct 30 13:18:05.126855 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:05.126827    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/07b7a046-d95c-4344-bbf9-4abb90b36fca-maas-api-xks-ca\") pod \"maas-api-revoke-sub-76f0e08b-t78dl\" (UID: \"07b7a046-d95c-4344-bbf9-4abb90b36fca\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-76f0e08b-t78dl"
Oct 30 13:18:05.126989 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:05.126884    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/07b7a046-d95c-4344-bbf9-4abb90b36fca-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-76f0e08b-t78dl\" (UID: \"07b7a046-d95c-4344-bbf9-4abb90b36fca\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-76f0e08b-t78dl"
Oct 30 13:18:05.127087 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:05.127070    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zvp8g\" (UniqueName: \"kubernetes.io/projected/07b7a046-d95c-4344-bbf9-4abb90b36fca-kube-api-access-zvp8g\") pod \"maas-api-revoke-sub-76f0e08b-t78dl\" (UID: \"07b7a046-d95c-4344-bbf9-4abb90b36fca\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-76f0e08b-t78dl"
Oct 30 13:18:05.229254 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:05.229222    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zvp8g\" (UniqueName: \"kubernetes.io/projected/07b7a046-d95c-4344-bbf9-4abb90b36fca-kube-api-access-zvp8g\") pod \"maas-api-revoke-sub-76f0e08b-t78dl\" (UID: \"07b7a046-d95c-4344-bbf9-4abb90b36fca\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-76f0e08b-t78dl"
Oct 30 13:18:05.229430 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:05.229351    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/07b7a046-d95c-4344-bbf9-4abb90b36fca-maas-api-xks-ca\") pod \"maas-api-revoke-sub-76f0e08b-t78dl\" (UID: \"07b7a046-d95c-4344-bbf9-4abb90b36fca\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-76f0e08b-t78dl"
Oct 30 13:18:05.229430 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:05.229415    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/07b7a046-d95c-4344-bbf9-4abb90b36fca-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-76f0e08b-t78dl\" (UID: \"07b7a046-d95c-4344-bbf9-4abb90b36fca\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-76f0e08b-t78dl"
Oct 30 13:18:05.230113 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:05.230085    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/07b7a046-d95c-4344-bbf9-4abb90b36fca-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-76f0e08b-t78dl\" (UID: \"07b7a046-d95c-4344-bbf9-4abb90b36fca\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-76f0e08b-t78dl"
Oct 30 13:18:05.231731 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:05.231713    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/07b7a046-d95c-4344-bbf9-4abb90b36fca-maas-api-xks-ca\") pod \"maas-api-revoke-sub-76f0e08b-t78dl\" (UID: \"07b7a046-d95c-4344-bbf9-4abb90b36fca\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-76f0e08b-t78dl"
Oct 30 13:18:05.254345 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:05.254319    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zvp8g\" (UniqueName: \"kubernetes.io/projected/07b7a046-d95c-4344-bbf9-4abb90b36fca-kube-api-access-zvp8g\") pod \"maas-api-revoke-sub-76f0e08b-t78dl\" (UID: \"07b7a046-d95c-4344-bbf9-4abb90b36fca\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-76f0e08b-t78dl"
Oct 30 13:18:05.339281 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:05.339173    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-76f0e08b-t78dl"
Oct 30 13:18:05.648167 ip-10-0-142-84 kubenswrapper[4943]: W1030 13:18:05.648124    4943 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod07b7a046_d95c_4344_bbf9_4abb90b36fca.slice/crio-1f21e3f53fc1bfe978a9b665448eeda19f01dfc1e16ad836a8d96e008fc5c615 WatchSource:0}: Error finding container 1f21e3f53fc1bfe978a9b665448eeda19f01dfc1e16ad836a8d96e008fc5c615: Status 404 returned error can't find the container with id 1f21e3f53fc1bfe978a9b665448eeda19f01dfc1e16ad836a8d96e008fc5c615
Oct 30 13:18:05.648711 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:05.648688    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-76f0e08b-t78dl"]
Oct 30 13:18:05.782546 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:05.782507    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-76f0e08b-t78dl" event={"ID":"07b7a046-d95c-4344-bbf9-4abb90b36fca","Type":"ContainerStarted","Data":"1f21e3f53fc1bfe978a9b665448eeda19f01dfc1e16ad836a8d96e008fc5c615"}
Oct 30 13:18:05.956662 ip-10-0-142-84 kubenswrapper[4943]: E1030 13:18:05.956616    4943 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-knqz6" containerName="extractor"
Oct 30 13:18:06.680989 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:06.680954    4943 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-dkztt"]
Oct 30 13:18:06.685465 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:06.685442    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-dkztt"
Oct 30 13:18:06.702836 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:06.702808    4943 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-w5n8g"]
Oct 30 13:18:06.708385 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:06.708366    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-w5n8g"
Oct 30 13:18:06.727463 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:06.727430    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-dkztt"]
Oct 30 13:18:06.770450 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:06.770403    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-w5n8g"]
Oct 30 13:18:06.799218 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:06.799178    4943 generic.go:356] "Generic (PLEG): container finished" podID="07b7a046-d95c-4344-bbf9-4abb90b36fca" containerID="b3cc5cfb534fb865001cbd1e0ab1e850f63c1142c25d6e5b6a199af6d7b8f740" exitCode=0
Oct 30 13:18:06.799454 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:06.799291    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-76f0e08b-t78dl" event={"ID":"07b7a046-d95c-4344-bbf9-4abb90b36fca","Type":"ContainerDied","Data":"b3cc5cfb534fb865001cbd1e0ab1e850f63c1142c25d6e5b6a199af6d7b8f740"}
Oct 30 13:18:06.858802 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:06.858769    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/126cf7ee-cae5-4afa-82f5-f36102a35b59-istio-token\") pod \"e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-dkztt\" (UID: \"126cf7ee-cae5-4afa-82f5-f36102a35b59\") " pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-dkztt"
Oct 30 13:18:06.858958 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:06.858864    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gj4s9\" (UniqueName: \"kubernetes.io/projected/126cf7ee-cae5-4afa-82f5-f36102a35b59-kube-api-access-gj4s9\") pod \"e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-dkztt\" (UID: \"126cf7ee-cae5-4afa-82f5-f36102a35b59\") " pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-dkztt"
Oct 30 13:18:06.858958 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:06.858899    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3eba0196-49e5-42ea-8192-ccaac19b1443-istiod-ca-cert\") pod \"e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-w5n8g\" (UID: \"3eba0196-49e5-42ea-8192-ccaac19b1443\") " pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-w5n8g"
Oct 30 13:18:06.858958 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:06.858929    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3eba0196-49e5-42ea-8192-ccaac19b1443-istio-data\") pod \"e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-w5n8g\" (UID: \"3eba0196-49e5-42ea-8192-ccaac19b1443\") " pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-w5n8g"
Oct 30 13:18:06.858958 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:06.858951    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/126cf7ee-cae5-4afa-82f5-f36102a35b59-istiod-ca-cert\") pod \"e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-dkztt\" (UID: \"126cf7ee-cae5-4afa-82f5-f36102a35b59\") " pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-dkztt"
Oct 30 13:18:06.859135 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:06.859057    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3eba0196-49e5-42ea-8192-ccaac19b1443-istio-token\") pod \"e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-w5n8g\" (UID: \"3eba0196-49e5-42ea-8192-ccaac19b1443\") " pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-w5n8g"
Oct 30 13:18:06.859135 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:06.859083    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3eba0196-49e5-42ea-8192-ccaac19b1443-istio-envoy\") pod \"e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-w5n8g\" (UID: \"3eba0196-49e5-42ea-8192-ccaac19b1443\") " pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-w5n8g"
Oct 30 13:18:06.859135 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:06.859116    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3eba0196-49e5-42ea-8192-ccaac19b1443-credential-socket\") pod \"e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-w5n8g\" (UID: \"3eba0196-49e5-42ea-8192-ccaac19b1443\") " pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-w5n8g"
Oct 30 13:18:06.859282 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:06.859142    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bkg9j\" (UniqueName: \"kubernetes.io/projected/3eba0196-49e5-42ea-8192-ccaac19b1443-kube-api-access-bkg9j\") pod \"e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-w5n8g\" (UID: \"3eba0196-49e5-42ea-8192-ccaac19b1443\") " pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-w5n8g"
Oct 30 13:18:06.859282 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:06.859182    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/126cf7ee-cae5-4afa-82f5-f36102a35b59-workload-socket\") pod \"e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-dkztt\" (UID: \"126cf7ee-cae5-4afa-82f5-f36102a35b59\") " pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-dkztt"
Oct 30 13:18:06.859394 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:06.859321    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/126cf7ee-cae5-4afa-82f5-f36102a35b59-credential-socket\") pod \"e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-dkztt\" (UID: \"126cf7ee-cae5-4afa-82f5-f36102a35b59\") " pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-dkztt"
Oct 30 13:18:06.859394 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:06.859354    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/126cf7ee-cae5-4afa-82f5-f36102a35b59-istio-envoy\") pod \"e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-dkztt\" (UID: \"126cf7ee-cae5-4afa-82f5-f36102a35b59\") " pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-dkztt"
Oct 30 13:18:06.859464 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:06.859413    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/126cf7ee-cae5-4afa-82f5-f36102a35b59-istio-podinfo\") pod \"e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-dkztt\" (UID: \"126cf7ee-cae5-4afa-82f5-f36102a35b59\") " pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-dkztt"
Oct 30 13:18:06.859499 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:06.859460    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/126cf7ee-cae5-4afa-82f5-f36102a35b59-istio-data\") pod \"e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-dkztt\" (UID: \"126cf7ee-cae5-4afa-82f5-f36102a35b59\") " pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-dkztt"
Oct 30 13:18:06.859499 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:06.859493    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3eba0196-49e5-42ea-8192-ccaac19b1443-istio-podinfo\") pod \"e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-w5n8g\" (UID: \"3eba0196-49e5-42ea-8192-ccaac19b1443\") " pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-w5n8g"
Oct 30 13:18:06.859591 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:06.859575    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/126cf7ee-cae5-4afa-82f5-f36102a35b59-workload-certs\") pod \"e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-dkztt\" (UID: \"126cf7ee-cae5-4afa-82f5-f36102a35b59\") " pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-dkztt"
Oct 30 13:18:06.859629 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:06.859619    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3eba0196-49e5-42ea-8192-ccaac19b1443-workload-certs\") pod \"e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-w5n8g\" (UID: \"3eba0196-49e5-42ea-8192-ccaac19b1443\") " pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-w5n8g"
Oct 30 13:18:06.859676 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:06.859647    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3eba0196-49e5-42ea-8192-ccaac19b1443-workload-socket\") pod \"e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-w5n8g\" (UID: \"3eba0196-49e5-42ea-8192-ccaac19b1443\") " pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-w5n8g"
Oct 30 13:18:06.960335 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:06.960304    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3eba0196-49e5-42ea-8192-ccaac19b1443-istio-data\") pod \"e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-w5n8g\" (UID: \"3eba0196-49e5-42ea-8192-ccaac19b1443\") " pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-w5n8g"
Oct 30 13:18:06.960517 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:06.960345    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/126cf7ee-cae5-4afa-82f5-f36102a35b59-istiod-ca-cert\") pod \"e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-dkztt\" (UID: \"126cf7ee-cae5-4afa-82f5-f36102a35b59\") " pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-dkztt"
Oct 30 13:18:06.960517 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:06.960392    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3eba0196-49e5-42ea-8192-ccaac19b1443-istio-token\") pod \"e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-w5n8g\" (UID: \"3eba0196-49e5-42ea-8192-ccaac19b1443\") " pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-w5n8g"
Oct 30 13:18:06.960517 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:06.960416    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3eba0196-49e5-42ea-8192-ccaac19b1443-istio-envoy\") pod \"e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-w5n8g\" (UID: \"3eba0196-49e5-42ea-8192-ccaac19b1443\") " pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-w5n8g"
Oct 30 13:18:06.960517 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:06.960452    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3eba0196-49e5-42ea-8192-ccaac19b1443-credential-socket\") pod \"e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-w5n8g\" (UID: \"3eba0196-49e5-42ea-8192-ccaac19b1443\") " pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-w5n8g"
Oct 30 13:18:06.960730 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:06.960595    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bkg9j\" (UniqueName: \"kubernetes.io/projected/3eba0196-49e5-42ea-8192-ccaac19b1443-kube-api-access-bkg9j\") pod \"e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-w5n8g\" (UID: \"3eba0196-49e5-42ea-8192-ccaac19b1443\") " pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-w5n8g"
Oct 30 13:18:06.960730 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:06.960631    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/126cf7ee-cae5-4afa-82f5-f36102a35b59-workload-socket\") pod \"e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-dkztt\" (UID: \"126cf7ee-cae5-4afa-82f5-f36102a35b59\") " pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-dkztt"
Oct 30 13:18:06.960730 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:06.960664    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/126cf7ee-cae5-4afa-82f5-f36102a35b59-credential-socket\") pod \"e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-dkztt\" (UID: \"126cf7ee-cae5-4afa-82f5-f36102a35b59\") " pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-dkztt"
Oct 30 13:18:06.960730 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:06.960689    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/126cf7ee-cae5-4afa-82f5-f36102a35b59-istio-envoy\") pod \"e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-dkztt\" (UID: \"126cf7ee-cae5-4afa-82f5-f36102a35b59\") " pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-dkztt"
Oct 30 13:18:06.960730 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:06.960720    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/126cf7ee-cae5-4afa-82f5-f36102a35b59-istio-podinfo\") pod \"e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-dkztt\" (UID: \"126cf7ee-cae5-4afa-82f5-f36102a35b59\") " pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-dkztt"
Oct 30 13:18:06.961015 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:06.960764    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3eba0196-49e5-42ea-8192-ccaac19b1443-istio-data\") pod \"e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-w5n8g\" (UID: \"3eba0196-49e5-42ea-8192-ccaac19b1443\") " pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-w5n8g"
Oct 30 13:18:06.961015 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:06.960775    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/126cf7ee-cae5-4afa-82f5-f36102a35b59-istio-data\") pod \"e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-dkztt\" (UID: \"126cf7ee-cae5-4afa-82f5-f36102a35b59\") " pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-dkztt"
Oct 30 13:18:06.961015 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:06.960806    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3eba0196-49e5-42ea-8192-ccaac19b1443-credential-socket\") pod \"e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-w5n8g\" (UID: \"3eba0196-49e5-42ea-8192-ccaac19b1443\") " pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-w5n8g"
Oct 30 13:18:06.961015 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:06.960837    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3eba0196-49e5-42ea-8192-ccaac19b1443-istio-podinfo\") pod \"e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-w5n8g\" (UID: \"3eba0196-49e5-42ea-8192-ccaac19b1443\") " pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-w5n8g"
Oct 30 13:18:06.961015 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:06.960896    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/126cf7ee-cae5-4afa-82f5-f36102a35b59-workload-certs\") pod \"e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-dkztt\" (UID: \"126cf7ee-cae5-4afa-82f5-f36102a35b59\") " pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-dkztt"
Oct 30 13:18:06.961015 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:06.960962    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3eba0196-49e5-42ea-8192-ccaac19b1443-workload-certs\") pod \"e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-w5n8g\" (UID: \"3eba0196-49e5-42ea-8192-ccaac19b1443\") " pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-w5n8g"
Oct 30 13:18:06.961015 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:06.961014    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3eba0196-49e5-42ea-8192-ccaac19b1443-workload-socket\") pod \"e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-w5n8g\" (UID: \"3eba0196-49e5-42ea-8192-ccaac19b1443\") " pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-w5n8g"
Oct 30 13:18:06.961401 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:06.961093    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/126cf7ee-cae5-4afa-82f5-f36102a35b59-credential-socket\") pod \"e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-dkztt\" (UID: \"126cf7ee-cae5-4afa-82f5-f36102a35b59\") " pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-dkztt"
Oct 30 13:18:06.961401 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:06.961112    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/126cf7ee-cae5-4afa-82f5-f36102a35b59-workload-socket\") pod \"e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-dkztt\" (UID: \"126cf7ee-cae5-4afa-82f5-f36102a35b59\") " pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-dkztt"
Oct 30 13:18:06.961401 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:06.961185    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/126cf7ee-cae5-4afa-82f5-f36102a35b59-istio-token\") pod \"e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-dkztt\" (UID: \"126cf7ee-cae5-4afa-82f5-f36102a35b59\") " pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-dkztt"
Oct 30 13:18:06.961401 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:06.961191    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/126cf7ee-cae5-4afa-82f5-f36102a35b59-istiod-ca-cert\") pod \"e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-dkztt\" (UID: \"126cf7ee-cae5-4afa-82f5-f36102a35b59\") " pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-dkztt"
Oct 30 13:18:06.961401 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:06.961197    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/126cf7ee-cae5-4afa-82f5-f36102a35b59-istio-data\") pod \"e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-dkztt\" (UID: \"126cf7ee-cae5-4afa-82f5-f36102a35b59\") " pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-dkztt"
Oct 30 13:18:06.961401 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:06.961254    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gj4s9\" (UniqueName: \"kubernetes.io/projected/126cf7ee-cae5-4afa-82f5-f36102a35b59-kube-api-access-gj4s9\") pod \"e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-dkztt\" (UID: \"126cf7ee-cae5-4afa-82f5-f36102a35b59\") " pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-dkztt"
Oct 30 13:18:06.961401 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:06.961332    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3eba0196-49e5-42ea-8192-ccaac19b1443-istiod-ca-cert\") pod \"e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-w5n8g\" (UID: \"3eba0196-49e5-42ea-8192-ccaac19b1443\") " pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-w5n8g"
Oct 30 13:18:06.961401 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:06.961395    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3eba0196-49e5-42ea-8192-ccaac19b1443-workload-socket\") pod \"e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-w5n8g\" (UID: \"3eba0196-49e5-42ea-8192-ccaac19b1443\") " pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-w5n8g"
Oct 30 13:18:06.961812 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:06.961517    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/126cf7ee-cae5-4afa-82f5-f36102a35b59-workload-certs\") pod \"e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-dkztt\" (UID: \"126cf7ee-cae5-4afa-82f5-f36102a35b59\") " pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-dkztt"
Oct 30 13:18:06.961812 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:06.961783    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3eba0196-49e5-42ea-8192-ccaac19b1443-workload-certs\") pod \"e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-w5n8g\" (UID: \"3eba0196-49e5-42ea-8192-ccaac19b1443\") " pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-w5n8g"
Oct 30 13:18:06.962046 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:06.962026    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3eba0196-49e5-42ea-8192-ccaac19b1443-istiod-ca-cert\") pod \"e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-w5n8g\" (UID: \"3eba0196-49e5-42ea-8192-ccaac19b1443\") " pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-w5n8g"
Oct 30 13:18:06.963498 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:06.963422    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3eba0196-49e5-42ea-8192-ccaac19b1443-istio-envoy\") pod \"e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-w5n8g\" (UID: \"3eba0196-49e5-42ea-8192-ccaac19b1443\") " pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-w5n8g"
Oct 30 13:18:06.963618 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:06.963584    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/126cf7ee-cae5-4afa-82f5-f36102a35b59-istio-envoy\") pod \"e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-dkztt\" (UID: \"126cf7ee-cae5-4afa-82f5-f36102a35b59\") " pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-dkztt"
Oct 30 13:18:06.963664 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:06.963642    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/126cf7ee-cae5-4afa-82f5-f36102a35b59-istio-podinfo\") pod \"e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-dkztt\" (UID: \"126cf7ee-cae5-4afa-82f5-f36102a35b59\") " pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-dkztt"
Oct 30 13:18:06.963900 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:06.963878    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3eba0196-49e5-42ea-8192-ccaac19b1443-istio-podinfo\") pod \"e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-w5n8g\" (UID: \"3eba0196-49e5-42ea-8192-ccaac19b1443\") " pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-w5n8g"
Oct 30 13:18:06.986225 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:06.986167    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3eba0196-49e5-42ea-8192-ccaac19b1443-istio-token\") pod \"e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-w5n8g\" (UID: \"3eba0196-49e5-42ea-8192-ccaac19b1443\") " pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-w5n8g"
Oct 30 13:18:06.986438 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:06.986413    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gj4s9\" (UniqueName: \"kubernetes.io/projected/126cf7ee-cae5-4afa-82f5-f36102a35b59-kube-api-access-gj4s9\") pod \"e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-dkztt\" (UID: \"126cf7ee-cae5-4afa-82f5-f36102a35b59\") " pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-dkztt"
Oct 30 13:18:06.986513 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:06.986499    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bkg9j\" (UniqueName: \"kubernetes.io/projected/3eba0196-49e5-42ea-8192-ccaac19b1443-kube-api-access-bkg9j\") pod \"e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-w5n8g\" (UID: \"3eba0196-49e5-42ea-8192-ccaac19b1443\") " pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-w5n8g"
Oct 30 13:18:06.988759 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:06.988739    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/126cf7ee-cae5-4afa-82f5-f36102a35b59-istio-token\") pod \"e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-dkztt\" (UID: \"126cf7ee-cae5-4afa-82f5-f36102a35b59\") " pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-dkztt"
Oct 30 13:18:07.001807 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:07.001784    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-dkztt"
Oct 30 13:18:07.028528 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:07.028504    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-w5n8g"
Oct 30 13:18:07.340592 ip-10-0-142-84 kubenswrapper[4943]: W1030 13:18:07.340558    4943 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod126cf7ee_cae5_4afa_82f5_f36102a35b59.slice/crio-f5cc62c06cbf340682f1a435ebbd8f4dfd089d98d1e0b3316a15b246d6bee208 WatchSource:0}: Error finding container f5cc62c06cbf340682f1a435ebbd8f4dfd089d98d1e0b3316a15b246d6bee208: Status 404 returned error can't find the container with id f5cc62c06cbf340682f1a435ebbd8f4dfd089d98d1e0b3316a15b246d6bee208
Oct 30 13:18:07.341315 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:07.341283    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-dkztt"]
Oct 30 13:18:07.343466 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:07.343431    4943 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 13:18:07.343553 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:07.343512    4943 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 13:18:07.343553 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:07.343540    4943 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 13:18:07.425197 ip-10-0-142-84 kubenswrapper[4943]: W1030 13:18:07.425171    4943 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3eba0196_49e5_42ea_8192_ccaac19b1443.slice/crio-70072baea518cb98cc23ae5ae41eaa2c8764ad46629f268cdc454cc801c8163a WatchSource:0}: Error finding container 70072baea518cb98cc23ae5ae41eaa2c8764ad46629f268cdc454cc801c8163a: Status 404 returned error can't find the container with id 70072baea518cb98cc23ae5ae41eaa2c8764ad46629f268cdc454cc801c8163a
Oct 30 13:18:07.433526 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:07.427117    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-w5n8g"]
Oct 30 13:18:07.433526 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:07.433188    4943 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 13:18:07.433526 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:07.433252    4943 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 13:18:07.433526 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:07.433308    4943 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 13:18:07.619310 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:07.617748    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-hdmrm"]
Oct 30 13:18:07.816798 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:07.816703    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-w5n8g" event={"ID":"3eba0196-49e5-42ea-8192-ccaac19b1443","Type":"ContainerStarted","Data":"65a941e112db7eca3ad6e91623540a02f99eceb72c59cea791506645ad7124a1"}
Oct 30 13:18:07.816798 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:07.816765    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-w5n8g" event={"ID":"3eba0196-49e5-42ea-8192-ccaac19b1443","Type":"ContainerStarted","Data":"70072baea518cb98cc23ae5ae41eaa2c8764ad46629f268cdc454cc801c8163a"}
Oct 30 13:18:07.818904 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:07.818860    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-dkztt" event={"ID":"126cf7ee-cae5-4afa-82f5-f36102a35b59","Type":"ContainerStarted","Data":"678f522b4439205a34af5a0823a35fd4af4579ba13f4fce57df24cfca5331fdc"}
Oct 30 13:18:07.819021 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:07.818930    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-dkztt" event={"ID":"126cf7ee-cae5-4afa-82f5-f36102a35b59","Type":"ContainerStarted","Data":"f5cc62c06cbf340682f1a435ebbd8f4dfd089d98d1e0b3316a15b246d6bee208"}
Oct 30 13:18:07.869307 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:07.868119    4943 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-w5n8g" podStartSLOduration=1.8681028149999999 podStartE2EDuration="1.868102815s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:18:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:18:07.865714016 +0000 UTC m=+1263.653717746" watchObservedRunningTime="2026-10-30 13:18:07.868102815 +0000 UTC m=+1263.656106625"
Oct 30 13:18:08.005293 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:08.005178    4943 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-dkztt"
Oct 30 13:18:08.012289 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:08.009365    4943 patch_prober.go:29] interesting pod/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-dkztt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.103:15021/healthz/ready\": dial tcp 10.132.0.103:15021: connect: connection refused" start-of-body=
Oct 30 13:18:08.012289 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:08.009424    4943 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-dkztt" podUID="126cf7ee-cae5-4afa-82f5-f36102a35b59" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.103:15021/healthz/ready\": dial tcp 10.132.0.103:15021: connect: connection refused"
Oct 30 13:18:08.029496 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:08.029454    4943 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-w5n8g"
Oct 30 13:18:08.030978 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:08.030917    4943 patch_prober.go:29] interesting pod/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-w5n8g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.104:15021/healthz/ready\": dial tcp 10.132.0.104:15021: connect: connection refused" start-of-body=
Oct 30 13:18:08.031184 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:08.030979    4943 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-w5n8g" podUID="3eba0196-49e5-42ea-8192-ccaac19b1443" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.104:15021/healthz/ready\": dial tcp 10.132.0.104:15021: connect: connection refused"
Oct 30 13:18:08.043931 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:08.043896    4943 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-76f0e08b-t78dl"
Oct 30 13:18:08.119038 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:08.116532    4943 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-dkztt" podStartSLOduration=2.11651003 podStartE2EDuration="2.11651003s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:18:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:18:07.924717093 +0000 UTC m=+1263.712720824" watchObservedRunningTime="2026-10-30 13:18:08.11651003 +0000 UTC m=+1263.904513761"
Oct 30 13:18:08.232838 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:08.230067    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/07b7a046-d95c-4344-bbf9-4abb90b36fca-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/07b7a046-d95c-4344-bbf9-4abb90b36fca-maas-api-openshift-ca\") pod \"07b7a046-d95c-4344-bbf9-4abb90b36fca\" (UID: \"07b7a046-d95c-4344-bbf9-4abb90b36fca\") "
Oct 30 13:18:08.232838 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:08.230188    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/07b7a046-d95c-4344-bbf9-4abb90b36fca-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/07b7a046-d95c-4344-bbf9-4abb90b36fca-maas-api-xks-ca\") pod \"07b7a046-d95c-4344-bbf9-4abb90b36fca\" (UID: \"07b7a046-d95c-4344-bbf9-4abb90b36fca\") "
Oct 30 13:18:08.232838 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:08.230389    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/07b7a046-d95c-4344-bbf9-4abb90b36fca-kube-api-access-zvp8g\" (UniqueName: \"kubernetes.io/projected/07b7a046-d95c-4344-bbf9-4abb90b36fca-kube-api-access-zvp8g\") pod \"07b7a046-d95c-4344-bbf9-4abb90b36fca\" (UID: \"07b7a046-d95c-4344-bbf9-4abb90b36fca\") "
Oct 30 13:18:08.232838 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:08.230627    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/07b7a046-d95c-4344-bbf9-4abb90b36fca-maas-api-openshift-ca" pod "07b7a046-d95c-4344-bbf9-4abb90b36fca" (UID: "07b7a046-d95c-4344-bbf9-4abb90b36fca"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 13:18:08.232838 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:08.231955    4943 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/07b7a046-d95c-4344-bbf9-4abb90b36fca-maas-api-openshift-ca\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:18:08.248235 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:08.248189    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/07b7a046-d95c-4344-bbf9-4abb90b36fca-maas-api-xks-ca" pod "07b7a046-d95c-4344-bbf9-4abb90b36fca" (UID: "07b7a046-d95c-4344-bbf9-4abb90b36fca"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 13:18:08.250413 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:08.250386    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/07b7a046-d95c-4344-bbf9-4abb90b36fca-kube-api-access-zvp8g" pod "07b7a046-d95c-4344-bbf9-4abb90b36fca" (UID: "07b7a046-d95c-4344-bbf9-4abb90b36fca"). InnerVolumeSpecName "kube-api-access-zvp8g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:18:08.333638 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:08.333593    4943 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/07b7a046-d95c-4344-bbf9-4abb90b36fca-maas-api-xks-ca\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:18:08.333638 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:08.333641    4943 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zvp8g\" (UniqueName: \"kubernetes.io/projected/07b7a046-d95c-4344-bbf9-4abb90b36fca-kube-api-access-zvp8g\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:18:08.839348 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:08.839310    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-76f0e08b-t78dl" event={"ID":"07b7a046-d95c-4344-bbf9-4abb90b36fca","Type":"ContainerDied","Data":"1f21e3f53fc1bfe978a9b665448eeda19f01dfc1e16ad836a8d96e008fc5c615"}
Oct 30 13:18:08.840877 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:08.840844    4943 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1f21e3f53fc1bfe978a9b665448eeda19f01dfc1e16ad836a8d96e008fc5c615"
Oct 30 13:18:08.841036 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:08.840423    4943 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-76f0e08b-t78dl"
Oct 30 13:18:09.003706 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:09.003670    4943 patch_prober.go:29] interesting pod/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-dkztt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.103:15021/healthz/ready\": dial tcp 10.132.0.103:15021: connect: connection refused" start-of-body=
Oct 30 13:18:09.003967 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:09.003938    4943 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-dkztt" podUID="126cf7ee-cae5-4afa-82f5-f36102a35b59" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.103:15021/healthz/ready\": dial tcp 10.132.0.103:15021: connect: connection refused"
Oct 30 13:18:09.036339 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:09.030699    4943 patch_prober.go:29] interesting pod/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-w5n8g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.104:15021/healthz/ready\": dial tcp 10.132.0.104:15021: connect: connection refused" start-of-body=
Oct 30 13:18:09.036339 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:09.030762    4943 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-w5n8g" podUID="3eba0196-49e5-42ea-8192-ccaac19b1443" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.104:15021/healthz/ready\": dial tcp 10.132.0.104:15021: connect: connection refused"
Oct 30 13:18:09.761592 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:09.759578    4943 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-05942d/distinct-2-w1-05942d-kserve-855c7f7c5f-pwt79" podUID="3dac65dc-7a67-4fdb-840a-765235aaa5bf" containerName="main" probeResult="failure" output="Get \"https://10.132.0.95:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:18:09.883769 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:09.881463    4943 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-f8a55e/distinct-w5-f8a55e-kserve-894fbd994-s2vsp" podUID="30f54ef4-28c2-4bf2-9feb-e2b0f1232c9f" containerName="main" probeResult="failure" output="Get \"https://10.132.0.92:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:18:10.013758 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:10.013661    4943 patch_prober.go:29] interesting pod/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-dkztt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.103:15021/healthz/ready\": dial tcp 10.132.0.103:15021: connect: connection refused" start-of-body=
Oct 30 13:18:10.019378 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:10.019333    4943 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-d0a18c/distinct-2-w3-d0a18c-kserve-85f8ff4846-rc44r" podUID="d65e47d2-4461-406a-9b71-bb3fac4262f0" containerName="main" probeResult="failure" output="Get \"https://10.132.0.93:8000/ready\": context deadline exceeded"
Oct 30 13:18:10.033598 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:10.033569    4943 patch_prober.go:29] interesting pod/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-w5n8g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.104:15021/healthz/ready\": dial tcp 10.132.0.104:15021: connect: connection refused" start-of-body=
Oct 30 13:18:10.033819 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:10.033795    4943 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-w5n8g" podUID="3eba0196-49e5-42ea-8192-ccaac19b1443" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.104:15021/healthz/ready\": dial tcp 10.132.0.104:15021: connect: connection refused"
Oct 30 13:18:10.041278 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:10.014842    4943 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-dkztt" podUID="126cf7ee-cae5-4afa-82f5-f36102a35b59" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.103:15021/healthz/ready\": dial tcp 10.132.0.103:15021: connect: connection refused"
Oct 30 13:18:10.366363 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:10.366242    4943 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w0-031737/premium-w0-031737-kserve-58d5857cdc-vfnz9" podUID="79b1a638-b019-4c89-b850-01699ecac9ba" containerName="main" probeResult="failure" output="Get \"https://10.132.0.70:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:18:10.575394 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:10.575346    4943 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-d0a18c/distinct-w3-d0a18c-kserve-86479f6bbf-5zcqq" podUID="63dace4c-7832-47dd-84ab-2c91483a5563" containerName="main" probeResult="failure" output="Get \"https://10.132.0.89:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:18:10.672506 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:10.672392    4943 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-031737/premium-w0-031737-kserve-58d5857cdc-vfnz9" podUID="79b1a638-b019-4c89-b850-01699ecac9ba" containerName="main" probeResult="failure" output="Get \"https://10.132.0.70:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:18:10.762469 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:10.762414    4943 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-05942d/distinct-2-w1-05942d-kserve-855c7f7c5f-pwt79" podUID="3dac65dc-7a67-4fdb-840a-765235aaa5bf" containerName="main" probeResult="failure" output="Get \"https://10.132.0.95:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:18:10.763648 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:10.763624    4943 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-05942d/distinct-2-w1-05942d-kserve-855c7f7c5f-pwt79"
Oct 30 13:18:10.792409 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:10.792371    4943 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-05942d/distinct-w1-05942d-kserve-5fb44dc664-mcbkc" podUID="d08eb359-cf9e-422c-a3a7-35fd173ab7b6" containerName="main" probeResult="failure" output="Get \"https://10.132.0.91:8000/ready\": context deadline exceeded"
Oct 30 13:18:10.884788 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:10.884748    4943 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-f8a55e/distinct-w5-f8a55e-kserve-894fbd994-s2vsp" podUID="30f54ef4-28c2-4bf2-9feb-e2b0f1232c9f" containerName="main" probeResult="failure" output="Get \"https://10.132.0.92:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:18:11.126311 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:11.023021    4943 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-d0a18c/distinct-2-w3-d0a18c-kserve-85f8ff4846-rc44r" podUID="d65e47d2-4461-406a-9b71-bb3fac4262f0" containerName="main" probeResult="failure" output="Get \"https://10.132.0.93:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:18:11.126311 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:11.117377    4943 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-f8a55e/distinct-w5-f8a55e-kserve-894fbd994-s2vsp"
Oct 30 13:18:11.126311 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:11.117443    4943 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-d0a18c/distinct-2-w3-d0a18c-kserve-85f8ff4846-rc44r"
Oct 30 13:18:11.126311 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:11.023386    4943 patch_prober.go:29] interesting pod/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-dkztt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.103:15021/healthz/ready\": dial tcp 10.132.0.103:15021: connect: connection refused" start-of-body=
Oct 30 13:18:11.126311 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:11.117821    4943 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-dkztt" podUID="126cf7ee-cae5-4afa-82f5-f36102a35b59" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.103:15021/healthz/ready\": dial tcp 10.132.0.103:15021: connect: connection refused"
Oct 30 13:18:11.126311 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:11.033971    4943 patch_prober.go:29] interesting pod/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-w5n8g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.104:15021/healthz/ready\": dial tcp 10.132.0.104:15021: connect: connection refused" start-of-body=
Oct 30 13:18:11.126311 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:11.118414    4943 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-w5n8g" podUID="3eba0196-49e5-42ea-8192-ccaac19b1443" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.104:15021/healthz/ready\": dial tcp 10.132.0.104:15021: connect: connection refused"
Oct 30 13:18:11.602036 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:11.593705    4943 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-ab3176/simulator-w6-ab3176-kserve-6d9b64549f-dc8xz" podUID="da306792-4f80-4df4-aa44-82991ab856e4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.69:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:18:11.628637 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:11.628589    4943 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-d0a18c/distinct-w3-d0a18c-kserve-86479f6bbf-5zcqq" podUID="63dace4c-7832-47dd-84ab-2c91483a5563" containerName="main" probeResult="failure" output="Get \"https://10.132.0.89:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:18:11.629438 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:11.629409    4943 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-d0a18c/distinct-w3-d0a18c-kserve-86479f6bbf-5zcqq"
Oct 30 13:18:11.680121 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:11.675599    4943 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-031737/premium-w0-031737-kserve-58d5857cdc-vfnz9" podUID="79b1a638-b019-4c89-b850-01699ecac9ba" containerName="main" probeResult="failure" output="Get \"https://10.132.0.70:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:18:11.680121 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:11.676479    4943 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-031737/premium-w0-031737-kserve-58d5857cdc-vfnz9"
Oct 30 13:18:11.787010 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:11.786940    4943 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-bc53a85a-dg5hj"]
Oct 30 13:18:11.801373 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:11.790160    4943 cpu_manager.go:414] "containerMap: removing container" podUID="07b7a046-d95c-4344-bbf9-4abb90b36fca" containerName="revoke-keys"
Oct 30 13:18:11.918881 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:11.918332    4943 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="07b7a046-d95c-4344-bbf9-4abb90b36fca" containerName="revoke-keys"
Oct 30 13:18:11.918881 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:11.900512    4943 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-05942d/distinct-2-w1-05942d-kserve-855c7f7c5f-pwt79" podUID="3dac65dc-7a67-4fdb-840a-765235aaa5bf" containerName="main" probeResult="failure" output="Get \"https://10.132.0.95:8000/ready\": context deadline exceeded"
Oct 30 13:18:11.918881 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:11.804423    4943 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-05942d/distinct-w1-05942d-kserve-5fb44dc664-mcbkc" podUID="d08eb359-cf9e-422c-a3a7-35fd173ab7b6" containerName="main" probeResult="failure" output="Get \"https://10.132.0.91:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:18:11.933034 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:11.933002    4943 memory_manager.go:365] "RemoveStaleState removing state" podUID="07b7a046-d95c-4344-bbf9-4abb90b36fca" containerName="revoke-keys"
Oct 30 13:18:12.157987 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:12.156460    4943 patch_prober.go:29] interesting pod/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-w5n8g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.104:15021/healthz/ready\": dial tcp 10.132.0.104:15021: connect: connection refused" start-of-body=
Oct 30 13:18:12.157987 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:12.156524    4943 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-w5n8g" podUID="3eba0196-49e5-42ea-8192-ccaac19b1443" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.104:15021/healthz/ready\": dial tcp 10.132.0.104:15021: connect: connection refused"
Oct 30 13:18:12.182366 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:12.181877    4943 patch_prober.go:29] interesting pod/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-dkztt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.103:15021/healthz/ready\": dial tcp 10.132.0.103:15021: connect: connection refused" start-of-body=
Oct 30 13:18:12.182366 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:12.181931    4943 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-dkztt" podUID="126cf7ee-cae5-4afa-82f5-f36102a35b59" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.103:15021/healthz/ready\": dial tcp 10.132.0.103:15021: connect: connection refused"
Oct 30 13:18:12.194812 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:12.187828    4943 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-d0a18c/distinct-2-w3-d0a18c-kserve-85f8ff4846-rc44r" podUID="d65e47d2-4461-406a-9b71-bb3fac4262f0" containerName="main" probeResult="failure" output="Get \"https://10.132.0.93:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:18:12.227282 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:12.189710    4943 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-f8a55e/distinct-w5-f8a55e-kserve-894fbd994-s2vsp" podUID="30f54ef4-28c2-4bf2-9feb-e2b0f1232c9f" containerName="main" probeResult="failure" output="Get \"https://10.132.0.92:8000/ready\": context deadline exceeded"
Oct 30 13:18:12.227282 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:12.195515    4943 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-05942d/distinct-w1-05942d-kserve-5fb44dc664-mcbkc"
Oct 30 13:18:12.227282 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:12.195905    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-bc53a85a-dg5hj"
Oct 30 13:18:12.254485 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:12.254451    4943 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 13:18:12.255144 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:12.255119    4943 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-p9z57\""
Oct 30 13:18:12.366290 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:12.360986    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-bc53a85a-dg5hj"]
Oct 30 13:18:12.448395 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:12.447249    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/b8812677-29e8-49cd-af4f-9dbb6e3658a8-maas-api-xks-ca\") pod \"maas-api-revoke-sub-bc53a85a-dg5hj\" (UID: \"b8812677-29e8-49cd-af4f-9dbb6e3658a8\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-bc53a85a-dg5hj"
Oct 30 13:18:12.448395 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:12.447467    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-frjvw\" (UniqueName: \"kubernetes.io/projected/b8812677-29e8-49cd-af4f-9dbb6e3658a8-kube-api-access-frjvw\") pod \"maas-api-revoke-sub-bc53a85a-dg5hj\" (UID: \"b8812677-29e8-49cd-af4f-9dbb6e3658a8\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-bc53a85a-dg5hj"
Oct 30 13:18:12.448395 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:12.447587    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/b8812677-29e8-49cd-af4f-9dbb6e3658a8-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-bc53a85a-dg5hj\" (UID: \"b8812677-29e8-49cd-af4f-9dbb6e3658a8\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-bc53a85a-dg5hj"
Oct 30 13:18:12.480311 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:12.479211    4943 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w3-d0a18c/distinct-w3-d0a18c-kserve-86479f6bbf-5zcqq" podUID="63dace4c-7832-47dd-84ab-2c91483a5563" containerName="main" probeResult="failure" output="Get \"https://10.132.0.89:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:18:12.594290 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:12.585157    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/b8812677-29e8-49cd-af4f-9dbb6e3658a8-maas-api-xks-ca\") pod \"maas-api-revoke-sub-bc53a85a-dg5hj\" (UID: \"b8812677-29e8-49cd-af4f-9dbb6e3658a8\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-bc53a85a-dg5hj"
Oct 30 13:18:12.594830 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:12.594787    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-frjvw\" (UniqueName: \"kubernetes.io/projected/b8812677-29e8-49cd-af4f-9dbb6e3658a8-kube-api-access-frjvw\") pod \"maas-api-revoke-sub-bc53a85a-dg5hj\" (UID: \"b8812677-29e8-49cd-af4f-9dbb6e3658a8\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-bc53a85a-dg5hj"
Oct 30 13:18:12.634181 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:12.631141    4943 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-d0a18c/distinct-w3-d0a18c-kserve-86479f6bbf-5zcqq" podUID="63dace4c-7832-47dd-84ab-2c91483a5563" containerName="main" probeResult="failure" output="Get \"https://10.132.0.89:8000/ready\": context deadline exceeded"
Oct 30 13:18:12.643988 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:12.639380    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/b8812677-29e8-49cd-af4f-9dbb6e3658a8-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-bc53a85a-dg5hj\" (UID: \"b8812677-29e8-49cd-af4f-9dbb6e3658a8\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-bc53a85a-dg5hj"
Oct 30 13:18:12.680117 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:12.680086    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-frjvw\" (UniqueName: \"kubernetes.io/projected/b8812677-29e8-49cd-af4f-9dbb6e3658a8-kube-api-access-frjvw\") pod \"maas-api-revoke-sub-bc53a85a-dg5hj\" (UID: \"b8812677-29e8-49cd-af4f-9dbb6e3658a8\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-bc53a85a-dg5hj"
Oct 30 13:18:12.681347 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:12.681312    4943 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-031737/premium-w0-031737-kserve-58d5857cdc-vfnz9" podUID="79b1a638-b019-4c89-b850-01699ecac9ba" containerName="main" probeResult="failure" output="Get \"https://10.132.0.70:8000/ready\": context deadline exceeded"
Oct 30 13:18:12.689511 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:12.689479    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/b8812677-29e8-49cd-af4f-9dbb6e3658a8-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-bc53a85a-dg5hj\" (UID: \"b8812677-29e8-49cd-af4f-9dbb6e3658a8\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-bc53a85a-dg5hj"
Oct 30 13:18:12.760346 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:12.746514    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/b8812677-29e8-49cd-af4f-9dbb6e3658a8-maas-api-xks-ca\") pod \"maas-api-revoke-sub-bc53a85a-dg5hj\" (UID: \"b8812677-29e8-49cd-af4f-9dbb6e3658a8\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-bc53a85a-dg5hj"
Oct 30 13:18:12.827780 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:12.826181    4943 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-f8a55e/distinct-w5-f8a55e-kserve-894fbd994-s2vsp"
Oct 30 13:18:12.886529 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:12.886499    4943 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-d0a18c/distinct-w3-d0a18c-kserve-86479f6bbf-5zcqq"
Oct 30 13:18:12.904524 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:12.904493    4943 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-d0a18c/distinct-2-w3-d0a18c-kserve-85f8ff4846-rc44r"
Oct 30 13:18:12.949507 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:12.949110    4943 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-031737/premium-w0-031737-kserve-58d5857cdc-vfnz9"
Oct 30 13:18:12.949507 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:12.949181    4943 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-05942d/distinct-2-w1-05942d-kserve-855c7f7c5f-pwt79"
Oct 30 13:18:12.950342 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:12.950317    4943 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-05942d/distinct-w1-05942d-kserve-5fb44dc664-mcbkc"
Oct 30 13:18:13.017736 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:13.017639    4943 patch_prober.go:29] interesting pod/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-dkztt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.103:15021/healthz/ready\": dial tcp 10.132.0.103:15021: connect: connection refused" start-of-body=
Oct 30 13:18:13.018045 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:13.017981    4943 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-dkztt" podUID="126cf7ee-cae5-4afa-82f5-f36102a35b59" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.103:15021/healthz/ready\": dial tcp 10.132.0.103:15021: connect: connection refused"
Oct 30 13:18:13.024303 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:13.020250    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-bc53a85a-dg5hj"
Oct 30 13:18:13.028824 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:13.028801    4943 patch_prober.go:29] interesting pod/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-w5n8g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.104:15021/healthz/ready\": dial tcp 10.132.0.104:15021: connect: connection refused" start-of-body=
Oct 30 13:18:13.028923 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:13.028853    4943 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-w5n8g" podUID="3eba0196-49e5-42ea-8192-ccaac19b1443" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.104:15021/healthz/ready\": dial tcp 10.132.0.104:15021: connect: connection refused"
Oct 30 13:18:13.608797 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:13.608762    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-bc53a85a-dg5hj"]
Oct 30 13:18:14.013624 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:14.011603    4943 patch_prober.go:29] interesting pod/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-dkztt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.103:15021/healthz/ready\": dial tcp 10.132.0.103:15021: connect: connection refused" start-of-body=
Oct 30 13:18:14.013624 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:14.011666    4943 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-dkztt" podUID="126cf7ee-cae5-4afa-82f5-f36102a35b59" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.103:15021/healthz/ready\": dial tcp 10.132.0.103:15021: connect: connection refused"
Oct 30 13:18:14.029367 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:14.029335    4943 patch_prober.go:29] interesting pod/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-w5n8g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.104:15021/healthz/ready\": dial tcp 10.132.0.104:15021: connect: connection refused" start-of-body=
Oct 30 13:18:14.029477 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:14.029377    4943 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-w5n8g" podUID="3eba0196-49e5-42ea-8192-ccaac19b1443" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.104:15021/healthz/ready\": dial tcp 10.132.0.104:15021: connect: connection refused"
Oct 30 13:18:14.616459 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:14.616403    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-bc53a85a-dg5hj" event={"ID":"b8812677-29e8-49cd-af4f-9dbb6e3658a8","Type":"ContainerStarted","Data":"577ce31b069a2df8ef798f2cba5c5b89aaf19002a7e45613e077e384c835c4ee"}
Oct 30 13:18:14.617003 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:14.616724    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-bc53a85a-dg5hj" event={"ID":"b8812677-29e8-49cd-af4f-9dbb6e3658a8","Type":"ContainerStarted","Data":"6d73fac8dfdd05c6a3326278b71bae3be07a86ded65b799447939347fc338289"}
Oct 30 13:18:14.704824 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:14.704759    4943 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-bc53a85a-dg5hj" podStartSLOduration=3.704739841 podStartE2EDuration="3.704739841s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:18:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:18:14.680541985 +0000 UTC m=+1270.468545729" watchObservedRunningTime="2026-10-30 13:18:14.704739841 +0000 UTC m=+1270.492743572"
Oct 30 13:18:15.006100 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:15.006062    4943 patch_prober.go:29] interesting pod/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-dkztt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.103:15021/healthz/ready\": dial tcp 10.132.0.103:15021: connect: connection refused" start-of-body=
Oct 30 13:18:15.008456 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:15.008422    4943 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-dkztt" podUID="126cf7ee-cae5-4afa-82f5-f36102a35b59" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.103:15021/healthz/ready\": dial tcp 10.132.0.103:15021: connect: connection refused"
Oct 30 13:18:15.040654 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:15.038659    4943 patch_prober.go:29] interesting pod/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-w5n8g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.104:15021/healthz/ready\": dial tcp 10.132.0.104:15021: connect: connection refused" start-of-body=
Oct 30 13:18:15.040654 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:15.038718    4943 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-w5n8g" podUID="3eba0196-49e5-42ea-8192-ccaac19b1443" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.104:15021/healthz/ready\": dial tcp 10.132.0.104:15021: connect: connection refused"
Oct 30 13:18:15.632284 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:15.631768    4943 generic.go:356] "Generic (PLEG): container finished" podID="b8812677-29e8-49cd-af4f-9dbb6e3658a8" containerID="577ce31b069a2df8ef798f2cba5c5b89aaf19002a7e45613e077e384c835c4ee" exitCode=0
Oct 30 13:18:15.632284 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:15.631870    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-bc53a85a-dg5hj" event={"ID":"b8812677-29e8-49cd-af4f-9dbb6e3658a8","Type":"ContainerDied","Data":"577ce31b069a2df8ef798f2cba5c5b89aaf19002a7e45613e077e384c835c4ee"}
Oct 30 13:18:16.002824 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:16.002792    4943 patch_prober.go:29] interesting pod/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-dkztt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.103:15021/healthz/ready\": dial tcp 10.132.0.103:15021: connect: connection refused" start-of-body=
Oct 30 13:18:16.002984 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:16.002853    4943 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-dkztt" podUID="126cf7ee-cae5-4afa-82f5-f36102a35b59" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.103:15021/healthz/ready\": dial tcp 10.132.0.103:15021: connect: connection refused"
Oct 30 13:18:16.028855 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:16.028821    4943 patch_prober.go:29] interesting pod/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-w5n8g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.104:15021/healthz/ready\": dial tcp 10.132.0.104:15021: connect: connection refused" start-of-body=
Oct 30 13:18:16.028998 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:16.028884    4943 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-w5n8g" podUID="3eba0196-49e5-42ea-8192-ccaac19b1443" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.104:15021/healthz/ready\": dial tcp 10.132.0.104:15021: connect: connection refused"
Oct 30 13:18:17.011630 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:17.010958    4943 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-bc53a85a-dg5hj"
Oct 30 13:18:17.096056 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:17.093975    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/b8812677-29e8-49cd-af4f-9dbb6e3658a8-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/b8812677-29e8-49cd-af4f-9dbb6e3658a8-maas-api-xks-ca\") pod \"b8812677-29e8-49cd-af4f-9dbb6e3658a8\" (UID: \"b8812677-29e8-49cd-af4f-9dbb6e3658a8\") "
Oct 30 13:18:17.096056 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:17.094102    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b8812677-29e8-49cd-af4f-9dbb6e3658a8-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/b8812677-29e8-49cd-af4f-9dbb6e3658a8-maas-api-openshift-ca\") pod \"b8812677-29e8-49cd-af4f-9dbb6e3658a8\" (UID: \"b8812677-29e8-49cd-af4f-9dbb6e3658a8\") "
Oct 30 13:18:17.096056 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:17.094156    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b8812677-29e8-49cd-af4f-9dbb6e3658a8-kube-api-access-frjvw\" (UniqueName: \"kubernetes.io/projected/b8812677-29e8-49cd-af4f-9dbb6e3658a8-kube-api-access-frjvw\") pod \"b8812677-29e8-49cd-af4f-9dbb6e3658a8\" (UID: \"b8812677-29e8-49cd-af4f-9dbb6e3658a8\") "
Oct 30 13:18:17.104287 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:17.097926    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b8812677-29e8-49cd-af4f-9dbb6e3658a8-maas-api-openshift-ca" pod "b8812677-29e8-49cd-af4f-9dbb6e3658a8" (UID: "b8812677-29e8-49cd-af4f-9dbb6e3658a8"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 13:18:17.140410 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:17.140375    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b8812677-29e8-49cd-af4f-9dbb6e3658a8-kube-api-access-frjvw" pod "b8812677-29e8-49cd-af4f-9dbb6e3658a8" (UID: "b8812677-29e8-49cd-af4f-9dbb6e3658a8"). InnerVolumeSpecName "kube-api-access-frjvw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:18:17.142412 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:17.142383    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b8812677-29e8-49cd-af4f-9dbb6e3658a8-maas-api-xks-ca" pod "b8812677-29e8-49cd-af4f-9dbb6e3658a8" (UID: "b8812677-29e8-49cd-af4f-9dbb6e3658a8"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 13:18:17.196370 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:17.196341    4943 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/b8812677-29e8-49cd-af4f-9dbb6e3658a8-maas-api-xks-ca\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:18:17.196370 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:17.196369    4943 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/b8812677-29e8-49cd-af4f-9dbb6e3658a8-maas-api-openshift-ca\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:18:17.196577 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:17.196380    4943 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-frjvw\" (UniqueName: \"kubernetes.io/projected/b8812677-29e8-49cd-af4f-9dbb6e3658a8-kube-api-access-frjvw\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:18:17.448611 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:17.448562    4943 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-w5n8g"
Oct 30 13:18:17.449310 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:17.449249    4943 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-w5n8g"
Oct 30 13:18:17.450630 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:17.450609    4943 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-w5n8g"
Oct 30 13:18:17.651926 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:17.651899    4943 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-bc53a85a-dg5hj"
Oct 30 13:18:17.652091 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:17.651923    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-bc53a85a-dg5hj" event={"ID":"b8812677-29e8-49cd-af4f-9dbb6e3658a8","Type":"ContainerDied","Data":"6d73fac8dfdd05c6a3326278b71bae3be07a86ded65b799447939347fc338289"}
Oct 30 13:18:17.652091 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:17.651975    4943 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6d73fac8dfdd05c6a3326278b71bae3be07a86ded65b799447939347fc338289"
Oct 30 13:18:17.695207 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:17.695174    4943 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-dkztt"
Oct 30 13:18:17.695704 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:17.695676    4943 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-dkztt"
Oct 30 13:18:17.696475 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:17.696459    4943 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-dkztt"
Oct 30 13:18:28.789498 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:28.789455    4943 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-62f1bc5f-klfbt"]
Oct 30 13:18:28.790564 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:28.790537    4943 cpu_manager.go:414] "containerMap: removing container" podUID="b8812677-29e8-49cd-af4f-9dbb6e3658a8" containerName="revoke-keys"
Oct 30 13:18:28.790564 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:28.790563    4943 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b8812677-29e8-49cd-af4f-9dbb6e3658a8" containerName="revoke-keys"
Oct 30 13:18:28.790750 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:28.790717    4943 memory_manager.go:365] "RemoveStaleState removing state" podUID="b8812677-29e8-49cd-af4f-9dbb6e3658a8" containerName="revoke-keys"
Oct 30 13:18:28.795379 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:28.795358    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-62f1bc5f-klfbt"
Oct 30 13:18:28.820020 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:28.819984    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-62f1bc5f-klfbt"]
Oct 30 13:18:28.827381 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:28.827357    4943 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 13:18:28.827568 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:28.827530    4943 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-p9z57\""
Oct 30 13:18:29.005286 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:28.999666    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ghkkp\" (UniqueName: \"kubernetes.io/projected/6918acda-77b9-4c03-86f0-c9ba259f80ae-kube-api-access-ghkkp\") pod \"maas-api-revoke-sub-62f1bc5f-klfbt\" (UID: \"6918acda-77b9-4c03-86f0-c9ba259f80ae\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-62f1bc5f-klfbt"
Oct 30 13:18:29.005286 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:28.999767    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/6918acda-77b9-4c03-86f0-c9ba259f80ae-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-62f1bc5f-klfbt\" (UID: \"6918acda-77b9-4c03-86f0-c9ba259f80ae\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-62f1bc5f-klfbt"
Oct 30 13:18:29.005286 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:28.999911    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/6918acda-77b9-4c03-86f0-c9ba259f80ae-maas-api-xks-ca\") pod \"maas-api-revoke-sub-62f1bc5f-klfbt\" (UID: \"6918acda-77b9-4c03-86f0-c9ba259f80ae\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-62f1bc5f-klfbt"
Oct 30 13:18:29.107341 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:29.107119    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/6918acda-77b9-4c03-86f0-c9ba259f80ae-maas-api-xks-ca\") pod \"maas-api-revoke-sub-62f1bc5f-klfbt\" (UID: \"6918acda-77b9-4c03-86f0-c9ba259f80ae\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-62f1bc5f-klfbt"
Oct 30 13:18:29.107511 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:29.107361    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ghkkp\" (UniqueName: \"kubernetes.io/projected/6918acda-77b9-4c03-86f0-c9ba259f80ae-kube-api-access-ghkkp\") pod \"maas-api-revoke-sub-62f1bc5f-klfbt\" (UID: \"6918acda-77b9-4c03-86f0-c9ba259f80ae\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-62f1bc5f-klfbt"
Oct 30 13:18:29.107511 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:29.107429    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/6918acda-77b9-4c03-86f0-c9ba259f80ae-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-62f1bc5f-klfbt\" (UID: \"6918acda-77b9-4c03-86f0-c9ba259f80ae\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-62f1bc5f-klfbt"
Oct 30 13:18:29.112282 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:29.108354    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/6918acda-77b9-4c03-86f0-c9ba259f80ae-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-62f1bc5f-klfbt\" (UID: \"6918acda-77b9-4c03-86f0-c9ba259f80ae\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-62f1bc5f-klfbt"
Oct 30 13:18:29.134303 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:29.123049    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/6918acda-77b9-4c03-86f0-c9ba259f80ae-maas-api-xks-ca\") pod \"maas-api-revoke-sub-62f1bc5f-klfbt\" (UID: \"6918acda-77b9-4c03-86f0-c9ba259f80ae\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-62f1bc5f-klfbt"
Oct 30 13:18:29.190289 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:29.175954    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ghkkp\" (UniqueName: \"kubernetes.io/projected/6918acda-77b9-4c03-86f0-c9ba259f80ae-kube-api-access-ghkkp\") pod \"maas-api-revoke-sub-62f1bc5f-klfbt\" (UID: \"6918acda-77b9-4c03-86f0-c9ba259f80ae\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-62f1bc5f-klfbt"
Oct 30 13:18:29.417335 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:29.416479    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-62f1bc5f-klfbt"
Oct 30 13:18:30.318222 ip-10-0-142-84 kubenswrapper[4943]: W1030 13:18:30.318187    4943 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6918acda_77b9_4c03_86f0_c9ba259f80ae.slice/crio-75e7a6f73d6758a018b97c0c3a1d7e9301574c6f5075bd0b61c49d12cfa0d71a WatchSource:0}: Error finding container 75e7a6f73d6758a018b97c0c3a1d7e9301574c6f5075bd0b61c49d12cfa0d71a: Status 404 returned error can't find the container with id 75e7a6f73d6758a018b97c0c3a1d7e9301574c6f5075bd0b61c49d12cfa0d71a
Oct 30 13:18:30.320219 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:30.320186    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-62f1bc5f-klfbt"]
Oct 30 13:18:30.857363 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:30.857312    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-62f1bc5f-klfbt" event={"ID":"6918acda-77b9-4c03-86f0-c9ba259f80ae","Type":"ContainerStarted","Data":"6c1ba9cd5851dfac47048825a8d9479ce683c6c7f2cdda92db7c5a375845c4be"}
Oct 30 13:18:30.857549 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:30.857379    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-62f1bc5f-klfbt" event={"ID":"6918acda-77b9-4c03-86f0-c9ba259f80ae","Type":"ContainerStarted","Data":"75e7a6f73d6758a018b97c0c3a1d7e9301574c6f5075bd0b61c49d12cfa0d71a"}
Oct 30 13:18:30.906698 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:30.906643    4943 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-62f1bc5f-klfbt" podStartSLOduration=2.9066285069999998 podStartE2EDuration="2.906628507s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:18:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:18:30.903181386 +0000 UTC m=+1286.691185117" watchObservedRunningTime="2026-10-30 13:18:30.906628507 +0000 UTC m=+1286.694632236"
Oct 30 13:18:31.866136 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:31.866094    4943 generic.go:356] "Generic (PLEG): container finished" podID="6918acda-77b9-4c03-86f0-c9ba259f80ae" containerID="6c1ba9cd5851dfac47048825a8d9479ce683c6c7f2cdda92db7c5a375845c4be" exitCode=0
Oct 30 13:18:31.866609 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:31.866175    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-62f1bc5f-klfbt" event={"ID":"6918acda-77b9-4c03-86f0-c9ba259f80ae","Type":"ContainerDied","Data":"6c1ba9cd5851dfac47048825a8d9479ce683c6c7f2cdda92db7c5a375845c4be"}
Oct 30 13:18:33.040463 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:33.040434    4943 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-62f1bc5f-klfbt"
Oct 30 13:18:33.145779 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:33.145680    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6918acda-77b9-4c03-86f0-c9ba259f80ae-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/6918acda-77b9-4c03-86f0-c9ba259f80ae-maas-api-openshift-ca\") pod \"6918acda-77b9-4c03-86f0-c9ba259f80ae\" (UID: \"6918acda-77b9-4c03-86f0-c9ba259f80ae\") "
Oct 30 13:18:33.159469 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:33.159421    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/6918acda-77b9-4c03-86f0-c9ba259f80ae-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/6918acda-77b9-4c03-86f0-c9ba259f80ae-maas-api-xks-ca\") pod \"6918acda-77b9-4c03-86f0-c9ba259f80ae\" (UID: \"6918acda-77b9-4c03-86f0-c9ba259f80ae\") "
Oct 30 13:18:33.180289 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:33.159676    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6918acda-77b9-4c03-86f0-c9ba259f80ae-maas-api-openshift-ca" pod "6918acda-77b9-4c03-86f0-c9ba259f80ae" (UID: "6918acda-77b9-4c03-86f0-c9ba259f80ae"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 13:18:33.194076 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:33.159712    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6918acda-77b9-4c03-86f0-c9ba259f80ae-kube-api-access-ghkkp\" (UniqueName: \"kubernetes.io/projected/6918acda-77b9-4c03-86f0-c9ba259f80ae-kube-api-access-ghkkp\") pod \"6918acda-77b9-4c03-86f0-c9ba259f80ae\" (UID: \"6918acda-77b9-4c03-86f0-c9ba259f80ae\") "
Oct 30 13:18:33.194663 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:33.194108    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6918acda-77b9-4c03-86f0-c9ba259f80ae-maas-api-xks-ca" pod "6918acda-77b9-4c03-86f0-c9ba259f80ae" (UID: "6918acda-77b9-4c03-86f0-c9ba259f80ae"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 13:18:33.197384 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:33.195488    4943 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/6918acda-77b9-4c03-86f0-c9ba259f80ae-maas-api-openshift-ca\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:18:33.197384 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:33.195523    4943 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/6918acda-77b9-4c03-86f0-c9ba259f80ae-maas-api-xks-ca\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:18:33.218736 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:33.218456    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6918acda-77b9-4c03-86f0-c9ba259f80ae-kube-api-access-ghkkp" pod "6918acda-77b9-4c03-86f0-c9ba259f80ae" (UID: "6918acda-77b9-4c03-86f0-c9ba259f80ae"). InnerVolumeSpecName "kube-api-access-ghkkp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:18:33.297220 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:33.297185    4943 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ghkkp\" (UniqueName: \"kubernetes.io/projected/6918acda-77b9-4c03-86f0-c9ba259f80ae-kube-api-access-ghkkp\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:18:33.882299 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:33.882256    4943 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-62f1bc5f-klfbt"
Oct 30 13:18:33.882472 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:33.882297    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-62f1bc5f-klfbt" event={"ID":"6918acda-77b9-4c03-86f0-c9ba259f80ae","Type":"ContainerDied","Data":"75e7a6f73d6758a018b97c0c3a1d7e9301574c6f5075bd0b61c49d12cfa0d71a"}
Oct 30 13:18:33.882472 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:33.882352    4943 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="75e7a6f73d6758a018b97c0c3a1d7e9301574c6f5075bd0b61c49d12cfa0d71a"
Oct 30 13:18:35.956253 ip-10-0-142-84 kubenswrapper[4943]: E1030 13:18:35.956212    4943 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-knqz6" containerName="extractor"
Oct 30 13:18:37.069184 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:37.069146    4943 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-58kq2"]
Oct 30 13:18:37.069938 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:37.069919    4943 cpu_manager.go:414] "containerMap: removing container" podUID="6918acda-77b9-4c03-86f0-c9ba259f80ae" containerName="revoke-keys"
Oct 30 13:18:37.070000 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:37.069940    4943 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6918acda-77b9-4c03-86f0-c9ba259f80ae" containerName="revoke-keys"
Oct 30 13:18:37.070048 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:37.070040    4943 memory_manager.go:365] "RemoveStaleState removing state" podUID="6918acda-77b9-4c03-86f0-c9ba259f80ae" containerName="revoke-keys"
Oct 30 13:18:37.074016 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:37.073996    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-58kq2"
Oct 30 13:18:37.118919 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:37.118886    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-58kq2"]
Oct 30 13:18:37.155162 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:37.155123    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0b3caf04-1a9d-44ce-8d80-3335b28a36ab-workload-socket\") pod \"e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-58kq2\" (UID: \"0b3caf04-1a9d-44ce-8d80-3335b28a36ab\") " pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-58kq2"
Oct 30 13:18:37.155387 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:37.155328    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0b3caf04-1a9d-44ce-8d80-3335b28a36ab-istio-podinfo\") pod \"e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-58kq2\" (UID: \"0b3caf04-1a9d-44ce-8d80-3335b28a36ab\") " pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-58kq2"
Oct 30 13:18:37.155471 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:37.155386    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0b3caf04-1a9d-44ce-8d80-3335b28a36ab-credential-socket\") pod \"e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-58kq2\" (UID: \"0b3caf04-1a9d-44ce-8d80-3335b28a36ab\") " pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-58kq2"
Oct 30 13:18:37.155471 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:37.155454    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0b3caf04-1a9d-44ce-8d80-3335b28a36ab-istio-token\") pod \"e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-58kq2\" (UID: \"0b3caf04-1a9d-44ce-8d80-3335b28a36ab\") " pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-58kq2"
Oct 30 13:18:37.155631 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:37.155593    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0b3caf04-1a9d-44ce-8d80-3335b28a36ab-istiod-ca-cert\") pod \"e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-58kq2\" (UID: \"0b3caf04-1a9d-44ce-8d80-3335b28a36ab\") " pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-58kq2"
Oct 30 13:18:37.155695 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:37.155654    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nxchf\" (UniqueName: \"kubernetes.io/projected/0b3caf04-1a9d-44ce-8d80-3335b28a36ab-kube-api-access-nxchf\") pod \"e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-58kq2\" (UID: \"0b3caf04-1a9d-44ce-8d80-3335b28a36ab\") " pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-58kq2"
Oct 30 13:18:37.155776 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:37.155752    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0b3caf04-1a9d-44ce-8d80-3335b28a36ab-workload-certs\") pod \"e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-58kq2\" (UID: \"0b3caf04-1a9d-44ce-8d80-3335b28a36ab\") " pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-58kq2"
Oct 30 13:18:37.155837 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:37.155815    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0b3caf04-1a9d-44ce-8d80-3335b28a36ab-istio-data\") pod \"e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-58kq2\" (UID: \"0b3caf04-1a9d-44ce-8d80-3335b28a36ab\") " pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-58kq2"
Oct 30 13:18:37.155901 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:37.155863    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0b3caf04-1a9d-44ce-8d80-3335b28a36ab-istio-envoy\") pod \"e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-58kq2\" (UID: \"0b3caf04-1a9d-44ce-8d80-3335b28a36ab\") " pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-58kq2"
Oct 30 13:18:37.257519 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:37.257487    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0b3caf04-1a9d-44ce-8d80-3335b28a36ab-istiod-ca-cert\") pod \"e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-58kq2\" (UID: \"0b3caf04-1a9d-44ce-8d80-3335b28a36ab\") " pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-58kq2"
Oct 30 13:18:37.257519 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:37.257522    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nxchf\" (UniqueName: \"kubernetes.io/projected/0b3caf04-1a9d-44ce-8d80-3335b28a36ab-kube-api-access-nxchf\") pod \"e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-58kq2\" (UID: \"0b3caf04-1a9d-44ce-8d80-3335b28a36ab\") " pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-58kq2"
Oct 30 13:18:37.257742 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:37.257560    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0b3caf04-1a9d-44ce-8d80-3335b28a36ab-workload-certs\") pod \"e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-58kq2\" (UID: \"0b3caf04-1a9d-44ce-8d80-3335b28a36ab\") " pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-58kq2"
Oct 30 13:18:37.257742 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:37.257598    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0b3caf04-1a9d-44ce-8d80-3335b28a36ab-istio-data\") pod \"e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-58kq2\" (UID: \"0b3caf04-1a9d-44ce-8d80-3335b28a36ab\") " pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-58kq2"
Oct 30 13:18:37.257742 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:37.257616    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0b3caf04-1a9d-44ce-8d80-3335b28a36ab-istio-envoy\") pod \"e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-58kq2\" (UID: \"0b3caf04-1a9d-44ce-8d80-3335b28a36ab\") " pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-58kq2"
Oct 30 13:18:37.257742 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:37.257652    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0b3caf04-1a9d-44ce-8d80-3335b28a36ab-workload-socket\") pod \"e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-58kq2\" (UID: \"0b3caf04-1a9d-44ce-8d80-3335b28a36ab\") " pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-58kq2"
Oct 30 13:18:37.257949 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:37.257788    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0b3caf04-1a9d-44ce-8d80-3335b28a36ab-istio-podinfo\") pod \"e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-58kq2\" (UID: \"0b3caf04-1a9d-44ce-8d80-3335b28a36ab\") " pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-58kq2"
Oct 30 13:18:37.257949 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:37.257833    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0b3caf04-1a9d-44ce-8d80-3335b28a36ab-credential-socket\") pod \"e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-58kq2\" (UID: \"0b3caf04-1a9d-44ce-8d80-3335b28a36ab\") " pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-58kq2"
Oct 30 13:18:37.257949 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:37.257926    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0b3caf04-1a9d-44ce-8d80-3335b28a36ab-istio-token\") pod \"e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-58kq2\" (UID: \"0b3caf04-1a9d-44ce-8d80-3335b28a36ab\") " pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-58kq2"
Oct 30 13:18:37.258107 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:37.258049    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0b3caf04-1a9d-44ce-8d80-3335b28a36ab-workload-certs\") pod \"e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-58kq2\" (UID: \"0b3caf04-1a9d-44ce-8d80-3335b28a36ab\") " pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-58kq2"
Oct 30 13:18:37.258370 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:37.258223    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0b3caf04-1a9d-44ce-8d80-3335b28a36ab-istio-data\") pod \"e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-58kq2\" (UID: \"0b3caf04-1a9d-44ce-8d80-3335b28a36ab\") " pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-58kq2"
Oct 30 13:18:37.258370 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:37.258336    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0b3caf04-1a9d-44ce-8d80-3335b28a36ab-istiod-ca-cert\") pod \"e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-58kq2\" (UID: \"0b3caf04-1a9d-44ce-8d80-3335b28a36ab\") " pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-58kq2"
Oct 30 13:18:37.258370 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:37.258338    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0b3caf04-1a9d-44ce-8d80-3335b28a36ab-workload-socket\") pod \"e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-58kq2\" (UID: \"0b3caf04-1a9d-44ce-8d80-3335b28a36ab\") " pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-58kq2"
Oct 30 13:18:37.258598 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:37.258466    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0b3caf04-1a9d-44ce-8d80-3335b28a36ab-credential-socket\") pod \"e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-58kq2\" (UID: \"0b3caf04-1a9d-44ce-8d80-3335b28a36ab\") " pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-58kq2"
Oct 30 13:18:37.260191 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:37.260170    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0b3caf04-1a9d-44ce-8d80-3335b28a36ab-istio-envoy\") pod \"e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-58kq2\" (UID: \"0b3caf04-1a9d-44ce-8d80-3335b28a36ab\") " pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-58kq2"
Oct 30 13:18:37.260679 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:37.260654    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0b3caf04-1a9d-44ce-8d80-3335b28a36ab-istio-podinfo\") pod \"e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-58kq2\" (UID: \"0b3caf04-1a9d-44ce-8d80-3335b28a36ab\") " pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-58kq2"
Oct 30 13:18:37.282819 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:37.282795    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0b3caf04-1a9d-44ce-8d80-3335b28a36ab-istio-token\") pod \"e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-58kq2\" (UID: \"0b3caf04-1a9d-44ce-8d80-3335b28a36ab\") " pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-58kq2"
Oct 30 13:18:37.283020 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:37.282997    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nxchf\" (UniqueName: \"kubernetes.io/projected/0b3caf04-1a9d-44ce-8d80-3335b28a36ab-kube-api-access-nxchf\") pod \"e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-58kq2\" (UID: \"0b3caf04-1a9d-44ce-8d80-3335b28a36ab\") " pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-58kq2"
Oct 30 13:18:37.389057 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:37.388996    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-58kq2"
Oct 30 13:18:37.719725 ip-10-0-142-84 kubenswrapper[4943]: W1030 13:18:37.719692    4943 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0b3caf04_1a9d_44ce_8d80_3335b28a36ab.slice/crio-165834ba1cc2100fdd411cacd92ddd10641d217fbd15c3bd409cd3aaa8e2eca3 WatchSource:0}: Error finding container 165834ba1cc2100fdd411cacd92ddd10641d217fbd15c3bd409cd3aaa8e2eca3: Status 404 returned error can't find the container with id 165834ba1cc2100fdd411cacd92ddd10641d217fbd15c3bd409cd3aaa8e2eca3
Oct 30 13:18:37.720644 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:37.720623    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-58kq2"]
Oct 30 13:18:37.722170 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:37.722138    4943 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 13:18:37.722225 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:37.722206    4943 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 13:18:37.722286 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:37.722238    4943 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 13:18:37.930281 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:37.929354    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-58kq2" event={"ID":"0b3caf04-1a9d-44ce-8d80-3335b28a36ab","Type":"ContainerStarted","Data":"82312698550d72f0e0cc6943d93c514b690a9e6ebc12f5d447d6cb49a5ba1d5b"}
Oct 30 13:18:37.930281 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:37.929452    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-58kq2" event={"ID":"0b3caf04-1a9d-44ce-8d80-3335b28a36ab","Type":"ContainerStarted","Data":"165834ba1cc2100fdd411cacd92ddd10641d217fbd15c3bd409cd3aaa8e2eca3"}
Oct 30 13:18:37.997002 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:37.996910    4943 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-58kq2" podStartSLOduration=0.996895892 podStartE2EDuration="996.895892ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:18:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:18:37.980191084 +0000 UTC m=+1293.768194814" watchObservedRunningTime="2026-10-30 13:18:37.996895892 +0000 UTC m=+1293.784899622"
Oct 30 13:18:38.389988 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:38.389903    4943 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-58kq2"
Oct 30 13:18:38.391642 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:38.391616    4943 patch_prober.go:29] interesting pod/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-58kq2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.107:15021/healthz/ready\": dial tcp 10.132.0.107:15021: connect: connection refused" start-of-body=
Oct 30 13:18:38.391736 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:38.391690    4943 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-58kq2" podUID="0b3caf04-1a9d-44ce-8d80-3335b28a36ab" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.107:15021/healthz/ready\": dial tcp 10.132.0.107:15021: connect: connection refused"
Oct 30 13:18:39.026411 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:39.026370    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-hdmrm"]
Oct 30 13:18:39.389873 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:39.389774    4943 patch_prober.go:29] interesting pod/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-58kq2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.107:15021/healthz/ready\": dial tcp 10.132.0.107:15021: connect: connection refused" start-of-body=
Oct 30 13:18:39.389873 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:39.389860    4943 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-58kq2" podUID="0b3caf04-1a9d-44ce-8d80-3335b28a36ab" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.107:15021/healthz/ready\": dial tcp 10.132.0.107:15021: connect: connection refused"
Oct 30 13:18:40.389519 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:40.389476    4943 patch_prober.go:29] interesting pod/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-58kq2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.107:15021/healthz/ready\": dial tcp 10.132.0.107:15021: connect: connection refused" start-of-body=
Oct 30 13:18:40.389820 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:40.389785    4943 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-58kq2" podUID="0b3caf04-1a9d-44ce-8d80-3335b28a36ab" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.107:15021/healthz/ready\": dial tcp 10.132.0.107:15021: connect: connection refused"
Oct 30 13:18:41.213285 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:41.193248    4943 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-d0a18c/distinct-2-w3-d0a18c-kserve-85f8ff4846-rc44r" podUID="d65e47d2-4461-406a-9b71-bb3fac4262f0" containerName="main" probeResult="failure" output="Get \"https://10.132.0.93:8000/ready\": context deadline exceeded"
Oct 30 13:18:41.391689 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:41.391644    4943 patch_prober.go:29] interesting pod/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-58kq2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.107:15021/healthz/ready\": dial tcp 10.132.0.107:15021: connect: connection refused" start-of-body=
Oct 30 13:18:41.391945 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:41.391913    4943 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-58kq2" podUID="0b3caf04-1a9d-44ce-8d80-3335b28a36ab" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.107:15021/healthz/ready\": dial tcp 10.132.0.107:15021: connect: connection refused"
Oct 30 13:18:41.555458 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:41.555354    4943 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-ab3176/simulator-w6-ab3176-kserve-6d9b64549f-dc8xz" podUID="da306792-4f80-4df4-aa44-82991ab856e4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.69:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:18:41.695404 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:41.695354    4943 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-031737/premium-w0-031737-kserve-58d5857cdc-vfnz9" podUID="79b1a638-b019-4c89-b850-01699ecac9ba" containerName="main" probeResult="failure" output="Get \"https://10.132.0.70:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:18:41.829472 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:41.829365    4943 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-f8a55e/distinct-w5-f8a55e-kserve-894fbd994-s2vsp" podUID="30f54ef4-28c2-4bf2-9feb-e2b0f1232c9f" containerName="main" probeResult="failure" output="Get \"https://10.132.0.92:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:18:41.898924 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:41.898881    4943 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-d0a18c/distinct-w3-d0a18c-kserve-86479f6bbf-5zcqq" podUID="63dace4c-7832-47dd-84ab-2c91483a5563" containerName="main" probeResult="failure" output="Get \"https://10.132.0.89:8000/ready\": context deadline exceeded"
Oct 30 13:18:41.951302 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:41.949348    4943 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-05942d/distinct-2-w1-05942d-kserve-855c7f7c5f-pwt79" podUID="3dac65dc-7a67-4fdb-840a-765235aaa5bf" containerName="main" probeResult="failure" output="Get \"https://10.132.0.95:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:18:42.037287 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:42.033379    4943 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-05942d/distinct-w1-05942d-kserve-5fb44dc664-mcbkc" podUID="d08eb359-cf9e-422c-a3a7-35fd173ab7b6" containerName="main" probeResult="failure" output="Get \"https://10.132.0.91:8000/ready\": context deadline exceeded"
Oct 30 13:18:42.197284 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:42.196338    4943 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w3-d0a18c/distinct-w3-d0a18c-kserve-86479f6bbf-5zcqq" podUID="63dace4c-7832-47dd-84ab-2c91483a5563" containerName="main" probeResult="failure" output="Get \"https://10.132.0.89:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:18:42.222285 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:42.204335    4943 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-d0a18c/distinct-2-w3-d0a18c-kserve-85f8ff4846-rc44r" podUID="d65e47d2-4461-406a-9b71-bb3fac4262f0" containerName="main" probeResult="failure" output="Get \"https://10.132.0.93:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:18:42.222285 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:42.204752    4943 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-d0a18c/distinct-2-w3-d0a18c-kserve-85f8ff4846-rc44r"
Oct 30 13:18:42.465340 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:42.464611    4943 patch_prober.go:29] interesting pod/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-58kq2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.107:15021/healthz/ready\": dial tcp 10.132.0.107:15021: connect: connection refused" start-of-body=
Oct 30 13:18:42.465340 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:42.465063    4943 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-58kq2" podUID="0b3caf04-1a9d-44ce-8d80-3335b28a36ab" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.107:15021/healthz/ready\": dial tcp 10.132.0.107:15021: connect: connection refused"
Oct 30 13:18:42.585482 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:42.569384    4943 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-ab3176/simulator-w6-ab3176-kserve-6d9b64549f-dc8xz" podUID="da306792-4f80-4df4-aa44-82991ab856e4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.69:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:18:42.705284 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:42.701546    4943 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-031737/premium-w0-031737-kserve-58d5857cdc-vfnz9" podUID="79b1a638-b019-4c89-b850-01699ecac9ba" containerName="main" probeResult="failure" output="Get \"https://10.132.0.70:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:18:42.804493 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:42.740435    4943 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-ab3176/simulator-w6-ab3176-kserve-6d9b64549f-dc8xz"
Oct 30 13:18:42.804884 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:42.804845    4943 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-031737/premium-w0-031737-kserve-58d5857cdc-vfnz9"
Oct 30 13:18:42.844287 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:42.832284    4943 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-f8a55e/distinct-w5-f8a55e-kserve-894fbd994-s2vsp" podUID="30f54ef4-28c2-4bf2-9feb-e2b0f1232c9f" containerName="main" probeResult="failure" output="Get \"https://10.132.0.92:8000/ready\": context deadline exceeded"
Oct 30 13:18:42.919533 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:42.919496    4943 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-f8a55e/distinct-w5-f8a55e-kserve-894fbd994-s2vsp"
Oct 30 13:18:42.952295 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:42.947477    4943 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-d0a18c/distinct-w3-d0a18c-kserve-86479f6bbf-5zcqq" podUID="63dace4c-7832-47dd-84ab-2c91483a5563" containerName="main" probeResult="failure" output="Get \"https://10.132.0.89:8000/ready\": context deadline exceeded"
Oct 30 13:18:43.031285 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:42.990036    4943 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-d0a18c/distinct-w3-d0a18c-kserve-86479f6bbf-5zcqq"
Oct 30 13:18:43.031285 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:42.990405    4943 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-05942d/distinct-2-w1-05942d-kserve-855c7f7c5f-pwt79" podUID="3dac65dc-7a67-4fdb-840a-765235aaa5bf" containerName="main" probeResult="failure" output="Get \"https://10.132.0.95:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:18:43.031285 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:42.990854    4943 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-05942d/distinct-2-w1-05942d-kserve-855c7f7c5f-pwt79"
Oct 30 13:18:43.053458 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:43.053413    4943 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-05942d/distinct-w1-05942d-kserve-5fb44dc664-mcbkc" podUID="d08eb359-cf9e-422c-a3a7-35fd173ab7b6" containerName="main" probeResult="failure" output="Get \"https://10.132.0.91:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:18:43.109513 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:43.084490    4943 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-05942d/distinct-w1-05942d-kserve-5fb44dc664-mcbkc"
Oct 30 13:18:43.141285 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:43.133630    4943 patch_prober.go:29] interesting pod/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-fdh7n container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 13:18:43.166447 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:43.166373    4943 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-fdh7n" podUID="8c4e0447-d0df-4536-bea2-00f13e370c5b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:18:43.237287 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:43.198985    4943 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w1-05942d/distinct-w1-05942d-kserve-5fb44dc664-mcbkc" podUID="d08eb359-cf9e-422c-a3a7-35fd173ab7b6" containerName="main" probeResult="failure" output="Get \"https://10.132.0.91:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:18:43.237287 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:43.210960    4943 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-d0a18c/distinct-2-w3-d0a18c-kserve-85f8ff4846-rc44r" podUID="d65e47d2-4461-406a-9b71-bb3fac4262f0" containerName="main" probeResult="failure" output="Get \"https://10.132.0.93:8000/ready\": context deadline exceeded"
Oct 30 13:18:43.497286 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:43.479683    4943 patch_prober.go:29] interesting pod/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-58kq2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.107:15021/healthz/ready\": dial tcp 10.132.0.107:15021: connect: connection refused" start-of-body=
Oct 30 13:18:43.497286 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:43.479806    4943 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-58kq2" podUID="0b3caf04-1a9d-44ce-8d80-3335b28a36ab" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.107:15021/healthz/ready\": dial tcp 10.132.0.107:15021: connect: connection refused"
Oct 30 13:18:43.581285 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:43.580852    4943 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-ab3176/simulator-w6-ab3176-kserve-6d9b64549f-dc8xz" podUID="da306792-4f80-4df4-aa44-82991ab856e4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.69:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:18:43.635733 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:43.635687    4943 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w3-d0a18c/distinct-2-w3-d0a18c-kserve-85f8ff4846-rc44r" podUID="d65e47d2-4461-406a-9b71-bb3fac4262f0" containerName="main" probeResult="failure" output="Get \"https://10.132.0.93:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:18:43.706229 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:43.706183    4943 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-031737/premium-w0-031737-kserve-58d5857cdc-vfnz9" podUID="79b1a638-b019-4c89-b850-01699ecac9ba" containerName="main" probeResult="failure" output="Get \"https://10.132.0.70:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:18:43.834639 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:43.834534    4943 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-f8a55e/distinct-w5-f8a55e-kserve-894fbd994-s2vsp" podUID="30f54ef4-28c2-4bf2-9feb-e2b0f1232c9f" containerName="main" probeResult="failure" output="Get \"https://10.132.0.92:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:18:43.877169 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:43.877138    4943 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-f8a55e/distinct-w5-f8a55e-kserve-894fbd994-s2vsp"
Oct 30 13:18:43.982386 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:43.982330    4943 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-d0a18c/distinct-w3-d0a18c-kserve-86479f6bbf-5zcqq" podUID="63dace4c-7832-47dd-84ab-2c91483a5563" containerName="main" probeResult="failure" output="Get \"https://10.132.0.89:8000/ready\": context deadline exceeded"
Oct 30 13:18:44.017293 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:44.003521    4943 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-05942d/distinct-2-w1-05942d-kserve-855c7f7c5f-pwt79" podUID="3dac65dc-7a67-4fdb-840a-765235aaa5bf" containerName="main" probeResult="failure" output="Get \"https://10.132.0.95:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:18:44.107381 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:44.098440    4943 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-05942d/distinct-w1-05942d-kserve-5fb44dc664-mcbkc" podUID="d08eb359-cf9e-422c-a3a7-35fd173ab7b6" containerName="main" probeResult="failure" output="Get \"https://10.132.0.91:8000/ready\": context deadline exceeded"
Oct 30 13:18:44.129981 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:44.129950    4943 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-ab3176/simulator-w6-ab3176-kserve-6d9b64549f-dc8xz"
Oct 30 13:18:44.184602 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:44.184568    4943 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-031737/premium-w0-031737-kserve-58d5857cdc-vfnz9"
Oct 30 13:18:44.188388 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:44.188364    4943 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-05942d/distinct-2-w1-05942d-kserve-855c7f7c5f-pwt79"
Oct 30 13:18:44.188605 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:44.188589    4943 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-d0a18c/distinct-w3-d0a18c-kserve-86479f6bbf-5zcqq"
Oct 30 13:18:44.215273 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:44.215219    4943 patch_prober.go:29] interesting pod/e2e-worker-w0-031737-openshift-default-64b8f46f47-zmddf container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.74:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 13:18:44.215851 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:44.215795    4943 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-zmddf" podUID="78fecfa2-495c-4690-bd0e-08e0a4213ade" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.74:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:18:44.221629 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:44.221586    4943 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-d0a18c/distinct-2-w3-d0a18c-kserve-85f8ff4846-rc44r" podUID="d65e47d2-4461-406a-9b71-bb3fac4262f0" containerName="main" probeResult="failure" output="Get \"https://10.132.0.93:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:18:44.389920 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:44.389836    4943 patch_prober.go:29] interesting pod/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-58kq2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.107:15021/healthz/ready\": dial tcp 10.132.0.107:15021: connect: connection refused" start-of-body=
Oct 30 13:18:44.392826 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:44.392791    4943 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-58kq2" podUID="0b3caf04-1a9d-44ce-8d80-3335b28a36ab" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.107:15021/healthz/ready\": dial tcp 10.132.0.107:15021: connect: connection refused"
Oct 30 13:18:44.473406 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:44.462413    4943 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-d0a18c/distinct-2-w3-d0a18c-kserve-85f8ff4846-rc44r"
Oct 30 13:18:44.473741 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:44.473718    4943 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-05942d/distinct-w1-05942d-kserve-5fb44dc664-mcbkc"
Oct 30 13:18:45.389646 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:45.389612    4943 patch_prober.go:29] interesting pod/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-58kq2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.107:15021/healthz/ready\": dial tcp 10.132.0.107:15021: connect: connection refused" start-of-body=
Oct 30 13:18:45.389831 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:45.389661    4943 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-58kq2" podUID="0b3caf04-1a9d-44ce-8d80-3335b28a36ab" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.107:15021/healthz/ready\": dial tcp 10.132.0.107:15021: connect: connection refused"
Oct 30 13:18:46.389952 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:46.389918    4943 patch_prober.go:29] interesting pod/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-58kq2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.107:15021/healthz/ready\": dial tcp 10.132.0.107:15021: connect: connection refused" start-of-body=
Oct 30 13:18:46.390656 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:46.389983    4943 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-58kq2" podUID="0b3caf04-1a9d-44ce-8d80-3335b28a36ab" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.107:15021/healthz/ready\": dial tcp 10.132.0.107:15021: connect: connection refused"
Oct 30 13:18:47.600172 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:47.600137    4943 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-58kq2"
Oct 30 13:18:47.601400 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:47.601370    4943 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-58kq2"
Oct 30 13:18:47.602039 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:47.602019    4943 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-58kq2"
Oct 30 13:18:49.526507 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:49.526473    4943 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-438019a3-qqgbp"]
Oct 30 13:18:49.530808 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:49.530786    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-438019a3-qqgbp"
Oct 30 13:18:49.552902 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:49.552873    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-438019a3-qqgbp"]
Oct 30 13:18:49.559450 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:49.559424    4943 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 13:18:49.559615 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:49.559473    4943 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-p9z57\""
Oct 30 13:18:49.613865 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:49.613827    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/b1b6f388-523f-4ec1-bf0d-4d31e1e6a6c6-maas-api-xks-ca\") pod \"maas-api-revoke-sub-438019a3-qqgbp\" (UID: \"b1b6f388-523f-4ec1-bf0d-4d31e1e6a6c6\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-438019a3-qqgbp"
Oct 30 13:18:49.614041 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:49.613963    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mpmjt\" (UniqueName: \"kubernetes.io/projected/b1b6f388-523f-4ec1-bf0d-4d31e1e6a6c6-kube-api-access-mpmjt\") pod \"maas-api-revoke-sub-438019a3-qqgbp\" (UID: \"b1b6f388-523f-4ec1-bf0d-4d31e1e6a6c6\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-438019a3-qqgbp"
Oct 30 13:18:49.614113 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:49.614067    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/b1b6f388-523f-4ec1-bf0d-4d31e1e6a6c6-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-438019a3-qqgbp\" (UID: \"b1b6f388-523f-4ec1-bf0d-4d31e1e6a6c6\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-438019a3-qqgbp"
Oct 30 13:18:49.715495 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:49.715463    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mpmjt\" (UniqueName: \"kubernetes.io/projected/b1b6f388-523f-4ec1-bf0d-4d31e1e6a6c6-kube-api-access-mpmjt\") pod \"maas-api-revoke-sub-438019a3-qqgbp\" (UID: \"b1b6f388-523f-4ec1-bf0d-4d31e1e6a6c6\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-438019a3-qqgbp"
Oct 30 13:18:49.715667 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:49.715553    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/b1b6f388-523f-4ec1-bf0d-4d31e1e6a6c6-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-438019a3-qqgbp\" (UID: \"b1b6f388-523f-4ec1-bf0d-4d31e1e6a6c6\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-438019a3-qqgbp"
Oct 30 13:18:49.715780 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:49.715764    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/b1b6f388-523f-4ec1-bf0d-4d31e1e6a6c6-maas-api-xks-ca\") pod \"maas-api-revoke-sub-438019a3-qqgbp\" (UID: \"b1b6f388-523f-4ec1-bf0d-4d31e1e6a6c6\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-438019a3-qqgbp"
Oct 30 13:18:49.716217 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:49.716195    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/b1b6f388-523f-4ec1-bf0d-4d31e1e6a6c6-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-438019a3-qqgbp\" (UID: \"b1b6f388-523f-4ec1-bf0d-4d31e1e6a6c6\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-438019a3-qqgbp"
Oct 30 13:18:49.718175 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:49.718157    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/b1b6f388-523f-4ec1-bf0d-4d31e1e6a6c6-maas-api-xks-ca\") pod \"maas-api-revoke-sub-438019a3-qqgbp\" (UID: \"b1b6f388-523f-4ec1-bf0d-4d31e1e6a6c6\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-438019a3-qqgbp"
Oct 30 13:18:49.740223 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:49.740199    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mpmjt\" (UniqueName: \"kubernetes.io/projected/b1b6f388-523f-4ec1-bf0d-4d31e1e6a6c6-kube-api-access-mpmjt\") pod \"maas-api-revoke-sub-438019a3-qqgbp\" (UID: \"b1b6f388-523f-4ec1-bf0d-4d31e1e6a6c6\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-438019a3-qqgbp"
Oct 30 13:18:49.846363 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:49.846295    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-438019a3-qqgbp"
Oct 30 13:18:50.152750 ip-10-0-142-84 kubenswrapper[4943]: W1030 13:18:50.152722    4943 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb1b6f388_523f_4ec1_bf0d_4d31e1e6a6c6.slice/crio-87cb23fe09a2e84062f6de4e7e2fa59e729fbb4381021baa640036fac31c5a87 WatchSource:0}: Error finding container 87cb23fe09a2e84062f6de4e7e2fa59e729fbb4381021baa640036fac31c5a87: Status 404 returned error can't find the container with id 87cb23fe09a2e84062f6de4e7e2fa59e729fbb4381021baa640036fac31c5a87
Oct 30 13:18:50.153432 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:50.153405    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-438019a3-qqgbp"]
Oct 30 13:18:50.590443 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:50.590393    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-438019a3-qqgbp" event={"ID":"b1b6f388-523f-4ec1-bf0d-4d31e1e6a6c6","Type":"ContainerStarted","Data":"1f4ae2ad3ab92bcb88f71224221e4563df821638db844dffca8f7204b8275b37"}
Oct 30 13:18:50.590848 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:50.590469    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-438019a3-qqgbp" event={"ID":"b1b6f388-523f-4ec1-bf0d-4d31e1e6a6c6","Type":"ContainerStarted","Data":"87cb23fe09a2e84062f6de4e7e2fa59e729fbb4381021baa640036fac31c5a87"}
Oct 30 13:18:50.638014 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:50.637944    4943 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-438019a3-qqgbp" podStartSLOduration=1.637926295 podStartE2EDuration="1.637926295s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:18:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:18:50.633765131 +0000 UTC m=+1306.421768872" watchObservedRunningTime="2026-10-30 13:18:50.637926295 +0000 UTC m=+1306.425930025"
Oct 30 13:18:51.599729 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:51.599694    4943 generic.go:356] "Generic (PLEG): container finished" podID="b1b6f388-523f-4ec1-bf0d-4d31e1e6a6c6" containerID="1f4ae2ad3ab92bcb88f71224221e4563df821638db844dffca8f7204b8275b37" exitCode=0
Oct 30 13:18:51.600155 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:51.599774    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-438019a3-qqgbp" event={"ID":"b1b6f388-523f-4ec1-bf0d-4d31e1e6a6c6","Type":"ContainerDied","Data":"1f4ae2ad3ab92bcb88f71224221e4563df821638db844dffca8f7204b8275b37"}
Oct 30 13:18:52.785146 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:52.785121    4943 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-438019a3-qqgbp"
Oct 30 13:18:52.888331 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:52.886127    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b1b6f388-523f-4ec1-bf0d-4d31e1e6a6c6-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/b1b6f388-523f-4ec1-bf0d-4d31e1e6a6c6-maas-api-openshift-ca\") pod \"b1b6f388-523f-4ec1-bf0d-4d31e1e6a6c6\" (UID: \"b1b6f388-523f-4ec1-bf0d-4d31e1e6a6c6\") "
Oct 30 13:18:52.888331 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:52.886334    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/b1b6f388-523f-4ec1-bf0d-4d31e1e6a6c6-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/b1b6f388-523f-4ec1-bf0d-4d31e1e6a6c6-maas-api-xks-ca\") pod \"b1b6f388-523f-4ec1-bf0d-4d31e1e6a6c6\" (UID: \"b1b6f388-523f-4ec1-bf0d-4d31e1e6a6c6\") "
Oct 30 13:18:52.888331 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:52.886467    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b1b6f388-523f-4ec1-bf0d-4d31e1e6a6c6-kube-api-access-mpmjt\" (UniqueName: \"kubernetes.io/projected/b1b6f388-523f-4ec1-bf0d-4d31e1e6a6c6-kube-api-access-mpmjt\") pod \"b1b6f388-523f-4ec1-bf0d-4d31e1e6a6c6\" (UID: \"b1b6f388-523f-4ec1-bf0d-4d31e1e6a6c6\") "
Oct 30 13:18:52.888331 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:52.887748    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b1b6f388-523f-4ec1-bf0d-4d31e1e6a6c6-maas-api-openshift-ca" pod "b1b6f388-523f-4ec1-bf0d-4d31e1e6a6c6" (UID: "b1b6f388-523f-4ec1-bf0d-4d31e1e6a6c6"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 13:18:52.894100 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:52.894072    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b1b6f388-523f-4ec1-bf0d-4d31e1e6a6c6-maas-api-xks-ca" pod "b1b6f388-523f-4ec1-bf0d-4d31e1e6a6c6" (UID: "b1b6f388-523f-4ec1-bf0d-4d31e1e6a6c6"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 13:18:52.894241 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:52.894210    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b1b6f388-523f-4ec1-bf0d-4d31e1e6a6c6-kube-api-access-mpmjt" pod "b1b6f388-523f-4ec1-bf0d-4d31e1e6a6c6" (UID: "b1b6f388-523f-4ec1-bf0d-4d31e1e6a6c6"). InnerVolumeSpecName "kube-api-access-mpmjt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:18:52.989069 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:52.989031    4943 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/b1b6f388-523f-4ec1-bf0d-4d31e1e6a6c6-maas-api-openshift-ca\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:18:52.989069 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:52.989069    4943 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/b1b6f388-523f-4ec1-bf0d-4d31e1e6a6c6-maas-api-xks-ca\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:18:52.989279 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:52.989086    4943 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mpmjt\" (UniqueName: \"kubernetes.io/projected/b1b6f388-523f-4ec1-bf0d-4d31e1e6a6c6-kube-api-access-mpmjt\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:18:53.620940 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:53.620905    4943 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-438019a3-qqgbp"
Oct 30 13:18:53.620940 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:53.620926    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-438019a3-qqgbp" event={"ID":"b1b6f388-523f-4ec1-bf0d-4d31e1e6a6c6","Type":"ContainerDied","Data":"87cb23fe09a2e84062f6de4e7e2fa59e729fbb4381021baa640036fac31c5a87"}
Oct 30 13:18:53.621194 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:53.620977    4943 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="87cb23fe09a2e84062f6de4e7e2fa59e729fbb4381021baa640036fac31c5a87"
Oct 30 13:18:58.124891 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:58.124847    4943 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-1699bfc5-9crvx"]
Oct 30 13:18:58.125556 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:58.125539    4943 cpu_manager.go:414] "containerMap: removing container" podUID="b1b6f388-523f-4ec1-bf0d-4d31e1e6a6c6" containerName="revoke-keys"
Oct 30 13:18:58.125556 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:58.125556    4943 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b1b6f388-523f-4ec1-bf0d-4d31e1e6a6c6" containerName="revoke-keys"
Oct 30 13:18:58.125671 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:58.125662    4943 memory_manager.go:365] "RemoveStaleState removing state" podUID="b1b6f388-523f-4ec1-bf0d-4d31e1e6a6c6" containerName="revoke-keys"
Oct 30 13:18:58.129429 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:58.129412    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-1699bfc5-9crvx"
Oct 30 13:18:58.154324 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:58.154298    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-1699bfc5-9crvx"]
Oct 30 13:18:58.162015 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:58.161993    4943 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 13:18:58.162189 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:58.162167    4943 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-p9z57\""
Oct 30 13:18:58.163810 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:58.163784    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/f7d7355b-c38e-49dd-8cd0-c5dd66306a90-maas-api-xks-ca\") pod \"maas-api-revoke-sub-1699bfc5-9crvx\" (UID: \"f7d7355b-c38e-49dd-8cd0-c5dd66306a90\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-1699bfc5-9crvx"
Oct 30 13:18:58.163914 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:58.163854    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/f7d7355b-c38e-49dd-8cd0-c5dd66306a90-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-1699bfc5-9crvx\" (UID: \"f7d7355b-c38e-49dd-8cd0-c5dd66306a90\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-1699bfc5-9crvx"
Oct 30 13:18:58.164005 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:58.163985    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jv94b\" (UniqueName: \"kubernetes.io/projected/f7d7355b-c38e-49dd-8cd0-c5dd66306a90-kube-api-access-jv94b\") pod \"maas-api-revoke-sub-1699bfc5-9crvx\" (UID: \"f7d7355b-c38e-49dd-8cd0-c5dd66306a90\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-1699bfc5-9crvx"
Oct 30 13:18:58.265369 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:58.265335    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jv94b\" (UniqueName: \"kubernetes.io/projected/f7d7355b-c38e-49dd-8cd0-c5dd66306a90-kube-api-access-jv94b\") pod \"maas-api-revoke-sub-1699bfc5-9crvx\" (UID: \"f7d7355b-c38e-49dd-8cd0-c5dd66306a90\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-1699bfc5-9crvx"
Oct 30 13:18:58.265537 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:58.265465    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/f7d7355b-c38e-49dd-8cd0-c5dd66306a90-maas-api-xks-ca\") pod \"maas-api-revoke-sub-1699bfc5-9crvx\" (UID: \"f7d7355b-c38e-49dd-8cd0-c5dd66306a90\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-1699bfc5-9crvx"
Oct 30 13:18:58.265537 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:58.265517    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/f7d7355b-c38e-49dd-8cd0-c5dd66306a90-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-1699bfc5-9crvx\" (UID: \"f7d7355b-c38e-49dd-8cd0-c5dd66306a90\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-1699bfc5-9crvx"
Oct 30 13:18:58.266138 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:58.266121    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/f7d7355b-c38e-49dd-8cd0-c5dd66306a90-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-1699bfc5-9crvx\" (UID: \"f7d7355b-c38e-49dd-8cd0-c5dd66306a90\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-1699bfc5-9crvx"
Oct 30 13:18:58.267817 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:58.267793    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/f7d7355b-c38e-49dd-8cd0-c5dd66306a90-maas-api-xks-ca\") pod \"maas-api-revoke-sub-1699bfc5-9crvx\" (UID: \"f7d7355b-c38e-49dd-8cd0-c5dd66306a90\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-1699bfc5-9crvx"
Oct 30 13:18:58.291353 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:58.291325    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jv94b\" (UniqueName: \"kubernetes.io/projected/f7d7355b-c38e-49dd-8cd0-c5dd66306a90-kube-api-access-jv94b\") pod \"maas-api-revoke-sub-1699bfc5-9crvx\" (UID: \"f7d7355b-c38e-49dd-8cd0-c5dd66306a90\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-1699bfc5-9crvx"
Oct 30 13:18:58.443906 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:58.443874    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-1699bfc5-9crvx"
Oct 30 13:18:58.814608 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:58.814576    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-1699bfc5-9crvx"]
Oct 30 13:18:59.670163 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:59.670125    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-1699bfc5-9crvx" event={"ID":"f7d7355b-c38e-49dd-8cd0-c5dd66306a90","Type":"ContainerStarted","Data":"bb01a7a097fcb1600c75cf6b3629fa2666873e502dadb07161f8bc8c12dfefd0"}
Oct 30 13:18:59.670612 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:59.670180    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-1699bfc5-9crvx" event={"ID":"f7d7355b-c38e-49dd-8cd0-c5dd66306a90","Type":"ContainerStarted","Data":"7d69a6b93d9a96cc6279b6feaed6f85e68bfd91866c4ae4092d40271101d83c4"}
Oct 30 13:18:59.717864 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:18:59.717811    4943 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-1699bfc5-9crvx" podStartSLOduration=1.7177986010000001 podStartE2EDuration="1.717798601s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:18:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:18:59.714932548 +0000 UTC m=+1315.502936293" watchObservedRunningTime="2026-10-30 13:18:59.717798601 +0000 UTC m=+1315.505802330"
Oct 30 13:19:00.681363 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:00.679508    4943 generic.go:356] "Generic (PLEG): container finished" podID="f7d7355b-c38e-49dd-8cd0-c5dd66306a90" containerID="bb01a7a097fcb1600c75cf6b3629fa2666873e502dadb07161f8bc8c12dfefd0" exitCode=0
Oct 30 13:19:00.681363 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:00.679713    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-1699bfc5-9crvx" event={"ID":"f7d7355b-c38e-49dd-8cd0-c5dd66306a90","Type":"ContainerDied","Data":"bb01a7a097fcb1600c75cf6b3629fa2666873e502dadb07161f8bc8c12dfefd0"}
Oct 30 13:19:01.221184 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:01.221002    4943 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w5-f8a55e/distinct-w5-f8a55e-kserve-894fbd994-s2vsp"]
Oct 30 13:19:02.887991 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:02.887933    4943 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-f8a55e/distinct-w5-f8a55e-kserve-894fbd994-s2vsp" podUID="30f54ef4-28c2-4bf2-9feb-e2b0f1232c9f" containerName="main" probeResult="failure" output="Get \"https://10.132.0.92:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:19:02.987584 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:02.987545    4943 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-d0a18c/distinct-w3-d0a18c-kserve-86479f6bbf-5zcqq" podUID="63dace4c-7832-47dd-84ab-2c91483a5563" containerName="main" probeResult="failure" output="Get \"https://10.132.0.89:8000/ready\": context deadline exceeded"
Oct 30 13:19:03.007778 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:03.007746    4943 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-1699bfc5-9crvx"
Oct 30 13:19:03.107288 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:03.103493    4943 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-05942d/distinct-w1-05942d-kserve-5fb44dc664-mcbkc" podUID="d08eb359-cf9e-422c-a3a7-35fd173ab7b6" containerName="main" probeResult="failure" output="Get \"https://10.132.0.91:8000/ready\": context deadline exceeded"
Oct 30 13:19:03.185365 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:03.160952    4943 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w1-05942d/distinct-w1-05942d-kserve-5fb44dc664-mcbkc" podUID="d08eb359-cf9e-422c-a3a7-35fd173ab7b6" containerName="main" probeResult="failure" output="Get \"https://10.132.0.91:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:19:03.189392 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:03.177401    4943 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-ab3176/simulator-w6-ab3176-kserve-6d9b64549f-dc8xz" podUID="da306792-4f80-4df4-aa44-82991ab856e4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.69:8000/ready\": context deadline exceeded"
Oct 30 13:19:03.215092 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:03.215035    4943 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-031737/premium-w0-031737-kserve-58d5857cdc-vfnz9" podUID="79b1a638-b019-4c89-b850-01699ecac9ba" containerName="main" probeResult="failure" output="Get \"https://10.132.0.70:8000/ready\": context deadline exceeded"
Oct 30 13:19:03.241329 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:03.214442    4943 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-05942d/distinct-2-w1-05942d-kserve-855c7f7c5f-pwt79" podUID="3dac65dc-7a67-4fdb-840a-765235aaa5bf" containerName="main" probeResult="failure" output="Get \"https://10.132.0.95:8000/ready\": context deadline exceeded"
Oct 30 13:19:03.241994 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:03.241967    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/f7d7355b-c38e-49dd-8cd0-c5dd66306a90-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/f7d7355b-c38e-49dd-8cd0-c5dd66306a90-maas-api-xks-ca\") pod \"f7d7355b-c38e-49dd-8cd0-c5dd66306a90\" (UID: \"f7d7355b-c38e-49dd-8cd0-c5dd66306a90\") "
Oct 30 13:19:03.242781 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:03.242343    4943 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-d0a18c/distinct-2-w3-d0a18c-kserve-85f8ff4846-rc44r" podUID="d65e47d2-4461-406a-9b71-bb3fac4262f0" containerName="main" probeResult="failure" output="Get \"https://10.132.0.93:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:19:03.243003 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:03.242735    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f7d7355b-c38e-49dd-8cd0-c5dd66306a90-kube-api-access-jv94b\" (UniqueName: \"kubernetes.io/projected/f7d7355b-c38e-49dd-8cd0-c5dd66306a90-kube-api-access-jv94b\") pod \"f7d7355b-c38e-49dd-8cd0-c5dd66306a90\" (UID: \"f7d7355b-c38e-49dd-8cd0-c5dd66306a90\") "
Oct 30 13:19:03.243319 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:03.243299    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f7d7355b-c38e-49dd-8cd0-c5dd66306a90-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/f7d7355b-c38e-49dd-8cd0-c5dd66306a90-maas-api-openshift-ca\") pod \"f7d7355b-c38e-49dd-8cd0-c5dd66306a90\" (UID: \"f7d7355b-c38e-49dd-8cd0-c5dd66306a90\") "
Oct 30 13:19:03.252530 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:03.252489    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f7d7355b-c38e-49dd-8cd0-c5dd66306a90-maas-api-openshift-ca" pod "f7d7355b-c38e-49dd-8cd0-c5dd66306a90" (UID: "f7d7355b-c38e-49dd-8cd0-c5dd66306a90"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 13:19:03.253478 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:03.253436    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f7d7355b-c38e-49dd-8cd0-c5dd66306a90-maas-api-xks-ca" pod "f7d7355b-c38e-49dd-8cd0-c5dd66306a90" (UID: "f7d7355b-c38e-49dd-8cd0-c5dd66306a90"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 13:19:03.289190 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:03.280722    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f7d7355b-c38e-49dd-8cd0-c5dd66306a90-kube-api-access-jv94b" pod "f7d7355b-c38e-49dd-8cd0-c5dd66306a90" (UID: "f7d7355b-c38e-49dd-8cd0-c5dd66306a90"). InnerVolumeSpecName "kube-api-access-jv94b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:19:03.349012 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:03.347302    4943 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/f7d7355b-c38e-49dd-8cd0-c5dd66306a90-maas-api-xks-ca\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:19:03.349012 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:03.347338    4943 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jv94b\" (UniqueName: \"kubernetes.io/projected/f7d7355b-c38e-49dd-8cd0-c5dd66306a90-kube-api-access-jv94b\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:19:03.349012 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:03.347353    4943 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/f7d7355b-c38e-49dd-8cd0-c5dd66306a90-maas-api-openshift-ca\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:19:03.506027 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:03.505926    4943 patch_prober.go:29] interesting pod/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-w5n8g container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.104:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 13:19:03.506320 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:03.506289    4943 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-w5n8g" podUID="3eba0196-49e5-42ea-8192-ccaac19b1443" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.104:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:19:03.546474 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:03.546361    4943 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w3-d0a18c/distinct-2-w3-d0a18c-kserve-85f8ff4846-rc44r" podUID="d65e47d2-4461-406a-9b71-bb3fac4262f0" containerName="main" probeResult="failure" output="Get \"https://10.132.0.93:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:19:03.861583 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:03.861479    4943 patch_prober.go:29] interesting pod/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-dkztt container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.103:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 13:19:03.861882 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:03.861832    4943 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-dkztt" podUID="126cf7ee-cae5-4afa-82f5-f36102a35b59" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.103:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:19:03.959424 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:03.959379    4943 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-f8a55e/distinct-w5-f8a55e-kserve-894fbd994-s2vsp" podUID="30f54ef4-28c2-4bf2-9feb-e2b0f1232c9f" containerName="main" probeResult="failure" output="Get \"https://10.132.0.92:8000/ready\": context deadline exceeded"
Oct 30 13:19:04.017285 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:04.013488    4943 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-f8a55e/distinct-w5-f8a55e-kserve-894fbd994-s2vsp"
Oct 30 13:19:04.017285 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:04.009362    4943 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-d0a18c/distinct-w3-d0a18c-kserve-86479f6bbf-5zcqq" podUID="63dace4c-7832-47dd-84ab-2c91483a5563" containerName="main" probeResult="failure" output="Get \"https://10.132.0.89:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:19:04.017285 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:04.007204    4943 patch_prober.go:29] interesting pod/e2e-worker-w3-d0a18c-openshift-default-547d57db97-9ndzw container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.101:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 13:19:04.017285 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:04.013855    4943 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-9ndzw" podUID="70c62fc1-3452-47f0-8d1f-5495e535cd95" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.101:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:19:04.017285 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:04.014217    4943 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-d0a18c/distinct-w3-d0a18c-kserve-86479f6bbf-5zcqq"
Oct 30 13:19:04.167393 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:04.118113    4943 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-05942d/distinct-w1-05942d-kserve-5fb44dc664-mcbkc" podUID="d08eb359-cf9e-422c-a3a7-35fd173ab7b6" containerName="main" probeResult="failure" output="Get \"https://10.132.0.91:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:19:04.192281 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:04.163369    4943 patch_prober.go:29] interesting pod/e2e-worker-w3-d0a18c-openshift-default-547d57db97-sfbdb container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.100:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 13:19:04.200284 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:04.192535    4943 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-sfbdb" podUID="4a720f44-f6a3-48b5-8ba4-04b3214486eb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.100:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:19:04.208283 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:04.200590    4943 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-05942d/distinct-w1-05942d-kserve-5fb44dc664-mcbkc"
Oct 30 13:19:04.233199 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:04.224953    4943 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-031737/premium-w0-031737-kserve-58d5857cdc-vfnz9" podUID="79b1a638-b019-4c89-b850-01699ecac9ba" containerName="main" probeResult="failure" output="Get \"https://10.132.0.70:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:19:04.261288 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:04.253524    4943 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-031737/premium-w0-031737-kserve-58d5857cdc-vfnz9"
Oct 30 13:19:04.261288 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:04.254317    4943 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-05942d/distinct-2-w1-05942d-kserve-855c7f7c5f-pwt79" podUID="3dac65dc-7a67-4fdb-840a-765235aaa5bf" containerName="main" probeResult="failure" output="Get \"https://10.132.0.95:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:19:04.261288 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:04.254690    4943 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-05942d/distinct-2-w1-05942d-kserve-855c7f7c5f-pwt79"
Oct 30 13:19:04.261288 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:04.259484    4943 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-d0a18c/distinct-2-w3-d0a18c-kserve-85f8ff4846-rc44r" podUID="d65e47d2-4461-406a-9b71-bb3fac4262f0" containerName="main" probeResult="failure" output="Get \"https://10.132.0.93:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:19:04.261288 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:04.259820    4943 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-d0a18c/distinct-2-w3-d0a18c-kserve-85f8ff4846-rc44r"
Oct 30 13:19:04.281283 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:04.280612    4943 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-ab3176/simulator-w6-ab3176-kserve-6d9b64549f-dc8xz" podUID="da306792-4f80-4df4-aa44-82991ab856e4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.69:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:19:04.281283 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:04.281026    4943 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-ab3176/simulator-w6-ab3176-kserve-6d9b64549f-dc8xz"
Oct 30 13:19:05.292286 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:05.272540    4943 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-05942d/distinct-2-w1-05942d-kserve-855c7f7c5f-pwt79" podUID="3dac65dc-7a67-4fdb-840a-765235aaa5bf" containerName="main" probeResult="failure" output="Get \"https://10.132.0.95:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:19:05.343006 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:05.329378    4943 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-d0a18c/distinct-w3-d0a18c-kserve-86479f6bbf-5zcqq" podUID="63dace4c-7832-47dd-84ab-2c91483a5563" containerName="main" probeResult="failure" output="Get \"https://10.132.0.89:8000/ready\": context deadline exceeded"
Oct 30 13:19:05.471074 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:05.471029    4943 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-ab3176/simulator-w6-ab3176-kserve-6d9b64549f-dc8xz" podUID="da306792-4f80-4df4-aa44-82991ab856e4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.69:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:19:05.560066 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:05.540155    4943 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-d0a18c/distinct-2-w3-d0a18c-kserve-85f8ff4846-rc44r" podUID="d65e47d2-4461-406a-9b71-bb3fac4262f0" containerName="main" probeResult="failure" output="Get \"https://10.132.0.93:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:19:05.603145 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:05.603100    4943 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-05942d/distinct-w1-05942d-kserve-5fb44dc664-mcbkc" podUID="d08eb359-cf9e-422c-a3a7-35fd173ab7b6" containerName="main" probeResult="failure" output="Get \"https://10.132.0.91:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:19:05.642881 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:05.634356    4943 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-031737/premium-w0-031737-kserve-58d5857cdc-vfnz9" podUID="79b1a638-b019-4c89-b850-01699ecac9ba" containerName="main" probeResult="failure" output="Get \"https://10.132.0.70:8000/ready\": context deadline exceeded"
Oct 30 13:19:05.980063 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:05.978725    4943 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w6-ab3176/simulator-w6-ab3176-kserve-6d9b64549f-dc8xz" podUID="da306792-4f80-4df4-aa44-82991ab856e4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.69:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:19:06.050138 ip-10-0-142-84 kubenswrapper[4943]: E1030 13:19:06.034530    4943 kubelet.go:2711] "Housekeeping took longer than expected" err="housekeeping took too long" expected="1s" actual="1.675s"
Oct 30 13:19:06.050138 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:06.034575    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-1699bfc5-9crvx" event={"ID":"f7d7355b-c38e-49dd-8cd0-c5dd66306a90","Type":"ContainerDied","Data":"7d69a6b93d9a96cc6279b6feaed6f85e68bfd91866c4ae4092d40271101d83c4"}
Oct 30 13:19:06.050138 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:06.034630    4943 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7d69a6b93d9a96cc6279b6feaed6f85e68bfd91866c4ae4092d40271101d83c4"
Oct 30 13:19:06.050138 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:06.035736    4943 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-1699bfc5-9crvx"
Oct 30 13:19:06.056626 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:06.034948    4943 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-f8a55e/distinct-w5-f8a55e-kserve-894fbd994-s2vsp" podUID="30f54ef4-28c2-4bf2-9feb-e2b0f1232c9f" containerName="main" probeResult="failure" output="Get \"https://10.132.0.92:8000/ready\": context deadline exceeded"
Oct 30 13:19:06.089336 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:06.089303    4943 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-d0a18c/distinct-2-w3-d0a18c-kserve-85f8ff4846-rc44r"
Oct 30 13:19:06.097760 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:06.097735    4943 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-031737/premium-w0-031737-kserve-58d5857cdc-vfnz9"
Oct 30 13:19:06.104556 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:06.102408    4943 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-d0a18c/distinct-w3-d0a18c-kserve-86479f6bbf-5zcqq"
Oct 30 13:19:06.104556 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:06.102465    4943 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-05942d/distinct-2-w1-05942d-kserve-855c7f7c5f-pwt79"
Oct 30 13:19:06.107529 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:06.107160    4943 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-f8a55e/distinct-w5-f8a55e-kserve-894fbd994-s2vsp"
Oct 30 13:19:06.115166 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:06.115137    4943 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-05942d/distinct-w1-05942d-kserve-5fb44dc664-mcbkc"
Oct 30 13:19:06.121745 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:06.121725    4943 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-ab3176/simulator-w6-ab3176-kserve-6d9b64549f-dc8xz"
Oct 30 13:19:06.144976 ip-10-0-142-84 kubenswrapper[4943]: E1030 13:19:06.144935    4943 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-knqz6" containerName="extractor"
Oct 30 13:19:06.723985 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:06.723947    4943 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-201d684c-rh5w4"]
Oct 30 13:19:06.724896 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:06.724872    4943 cpu_manager.go:414] "containerMap: removing container" podUID="f7d7355b-c38e-49dd-8cd0-c5dd66306a90" containerName="revoke-keys"
Oct 30 13:19:06.724945 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:06.724900    4943 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f7d7355b-c38e-49dd-8cd0-c5dd66306a90" containerName="revoke-keys"
Oct 30 13:19:06.725150 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:06.725135    4943 memory_manager.go:365] "RemoveStaleState removing state" podUID="f7d7355b-c38e-49dd-8cd0-c5dd66306a90" containerName="revoke-keys"
Oct 30 13:19:06.729373 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:06.729314    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-201d684c-rh5w4"
Oct 30 13:19:06.735046 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:06.735024    4943 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w5-f8a55e-697b9bfd75-pcr7h"]
Oct 30 13:19:06.743742 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:06.743719    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-201d684c-rh5w4"]
Oct 30 13:19:06.748852 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:06.748830    4943 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 13:19:06.748959 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:06.748896    4943 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-p9z57\""
Oct 30 13:19:06.834842 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:06.828787    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-trwnd\" (UniqueName: \"kubernetes.io/projected/6f0595d0-f37c-4067-b1e0-de986b38918b-kube-api-access-trwnd\") pod \"maas-api-revoke-sub-201d684c-rh5w4\" (UID: \"6f0595d0-f37c-4067-b1e0-de986b38918b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-201d684c-rh5w4"
Oct 30 13:19:06.834842 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:06.829065    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/6f0595d0-f37c-4067-b1e0-de986b38918b-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-201d684c-rh5w4\" (UID: \"6f0595d0-f37c-4067-b1e0-de986b38918b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-201d684c-rh5w4"
Oct 30 13:19:06.834842 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:06.829168    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/6f0595d0-f37c-4067-b1e0-de986b38918b-maas-api-xks-ca\") pod \"maas-api-revoke-sub-201d684c-rh5w4\" (UID: \"6f0595d0-f37c-4067-b1e0-de986b38918b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-201d684c-rh5w4"
Oct 30 13:19:06.930067 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:06.930034    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/6f0595d0-f37c-4067-b1e0-de986b38918b-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-201d684c-rh5w4\" (UID: \"6f0595d0-f37c-4067-b1e0-de986b38918b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-201d684c-rh5w4"
Oct 30 13:19:06.930299 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:06.930088    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/6f0595d0-f37c-4067-b1e0-de986b38918b-maas-api-xks-ca\") pod \"maas-api-revoke-sub-201d684c-rh5w4\" (UID: \"6f0595d0-f37c-4067-b1e0-de986b38918b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-201d684c-rh5w4"
Oct 30 13:19:06.930299 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:06.930204    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-trwnd\" (UniqueName: \"kubernetes.io/projected/6f0595d0-f37c-4067-b1e0-de986b38918b-kube-api-access-trwnd\") pod \"maas-api-revoke-sub-201d684c-rh5w4\" (UID: \"6f0595d0-f37c-4067-b1e0-de986b38918b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-201d684c-rh5w4"
Oct 30 13:19:06.930783 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:06.930755    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/6f0595d0-f37c-4067-b1e0-de986b38918b-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-201d684c-rh5w4\" (UID: \"6f0595d0-f37c-4067-b1e0-de986b38918b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-201d684c-rh5w4"
Oct 30 13:19:06.932620 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:06.932596    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/6f0595d0-f37c-4067-b1e0-de986b38918b-maas-api-xks-ca\") pod \"maas-api-revoke-sub-201d684c-rh5w4\" (UID: \"6f0595d0-f37c-4067-b1e0-de986b38918b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-201d684c-rh5w4"
Oct 30 13:19:06.995338 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:06.994951    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-trwnd\" (UniqueName: \"kubernetes.io/projected/6f0595d0-f37c-4067-b1e0-de986b38918b-kube-api-access-trwnd\") pod \"maas-api-revoke-sub-201d684c-rh5w4\" (UID: \"6f0595d0-f37c-4067-b1e0-de986b38918b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-201d684c-rh5w4"
Oct 30 13:19:07.053299 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:07.049501    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-201d684c-rh5w4"
Oct 30 13:19:07.395560 ip-10-0-142-84 kubenswrapper[4943]: W1030 13:19:07.395524    4943 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6f0595d0_f37c_4067_b1e0_de986b38918b.slice/crio-70370c5b02ec4443ce3d0e11235c22973c8427915e660bbd65ee9eac7f80baa1 WatchSource:0}: Error finding container 70370c5b02ec4443ce3d0e11235c22973c8427915e660bbd65ee9eac7f80baa1: Status 404 returned error can't find the container with id 70370c5b02ec4443ce3d0e11235c22973c8427915e660bbd65ee9eac7f80baa1
Oct 30 13:19:07.395792 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:07.395768    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-201d684c-rh5w4"]
Oct 30 13:19:07.788569 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:07.788531    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-201d684c-rh5w4" event={"ID":"6f0595d0-f37c-4067-b1e0-de986b38918b","Type":"ContainerStarted","Data":"70370c5b02ec4443ce3d0e11235c22973c8427915e660bbd65ee9eac7f80baa1"}
Oct 30 13:19:08.799059 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:08.799026    4943 generic.go:356] "Generic (PLEG): container finished" podID="6f0595d0-f37c-4067-b1e0-de986b38918b" containerID="2c16604c945c13324aa77fc889221b8fb2abd4deeed2dae7406028f881929db7" exitCode=0
Oct 30 13:19:08.799519 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:08.799083    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-201d684c-rh5w4" event={"ID":"6f0595d0-f37c-4067-b1e0-de986b38918b","Type":"ContainerDied","Data":"2c16604c945c13324aa77fc889221b8fb2abd4deeed2dae7406028f881929db7"}
Oct 30 13:19:09.961588 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:09.961564    4943 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-201d684c-rh5w4"
Oct 30 13:19:09.977520 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:09.977496    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/6f0595d0-f37c-4067-b1e0-de986b38918b-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/6f0595d0-f37c-4067-b1e0-de986b38918b-maas-api-xks-ca\") pod \"6f0595d0-f37c-4067-b1e0-de986b38918b\" (UID: \"6f0595d0-f37c-4067-b1e0-de986b38918b\") "
Oct 30 13:19:09.977642 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:09.977534    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6f0595d0-f37c-4067-b1e0-de986b38918b-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/6f0595d0-f37c-4067-b1e0-de986b38918b-maas-api-openshift-ca\") pod \"6f0595d0-f37c-4067-b1e0-de986b38918b\" (UID: \"6f0595d0-f37c-4067-b1e0-de986b38918b\") "
Oct 30 13:19:09.977710 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:09.977701    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6f0595d0-f37c-4067-b1e0-de986b38918b-kube-api-access-trwnd\" (UniqueName: \"kubernetes.io/projected/6f0595d0-f37c-4067-b1e0-de986b38918b-kube-api-access-trwnd\") pod \"6f0595d0-f37c-4067-b1e0-de986b38918b\" (UID: \"6f0595d0-f37c-4067-b1e0-de986b38918b\") "
Oct 30 13:19:09.977965 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:09.977932    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6f0595d0-f37c-4067-b1e0-de986b38918b-maas-api-openshift-ca" pod "6f0595d0-f37c-4067-b1e0-de986b38918b" (UID: "6f0595d0-f37c-4067-b1e0-de986b38918b"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 13:19:09.978341 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:09.978319    4943 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/6f0595d0-f37c-4067-b1e0-de986b38918b-maas-api-openshift-ca\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:19:09.979646 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:09.979620    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6f0595d0-f37c-4067-b1e0-de986b38918b-kube-api-access-trwnd" pod "6f0595d0-f37c-4067-b1e0-de986b38918b" (UID: "6f0595d0-f37c-4067-b1e0-de986b38918b"). InnerVolumeSpecName "kube-api-access-trwnd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:19:09.979646 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:09.979632    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6f0595d0-f37c-4067-b1e0-de986b38918b-maas-api-xks-ca" pod "6f0595d0-f37c-4067-b1e0-de986b38918b" (UID: "6f0595d0-f37c-4067-b1e0-de986b38918b"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 13:19:10.080049 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:10.079978    4943 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-trwnd\" (UniqueName: \"kubernetes.io/projected/6f0595d0-f37c-4067-b1e0-de986b38918b-kube-api-access-trwnd\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:19:10.080049 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:10.080005    4943 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/6f0595d0-f37c-4067-b1e0-de986b38918b-maas-api-xks-ca\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:19:10.817228 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:10.817139    4943 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-201d684c-rh5w4"
Oct 30 13:19:10.817228 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:10.817170    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-201d684c-rh5w4" event={"ID":"6f0595d0-f37c-4067-b1e0-de986b38918b","Type":"ContainerDied","Data":"70370c5b02ec4443ce3d0e11235c22973c8427915e660bbd65ee9eac7f80baa1"}
Oct 30 13:19:10.817228 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:10.817225    4943 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="70370c5b02ec4443ce3d0e11235c22973c8427915e660bbd65ee9eac7f80baa1"
Oct 30 13:19:11.735684 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:11.735645    4943 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-f8a55e-697b9bfd75-pcr7h" podUID="93484267-ce16-4ae2-b52b-e46fae5e0121" containerName="payload-pre-processing" containerID="cri-o://6391329407a4754f2b839e56094f340331c02341f487db984c4ed9943402c1f3" gracePeriod=25
Oct 30 13:19:11.805323 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:11.805292    4943 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-acd1d13c-sbxnv"]
Oct 30 13:19:11.805960 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:11.805944    4943 cpu_manager.go:414] "containerMap: removing container" podUID="6f0595d0-f37c-4067-b1e0-de986b38918b" containerName="revoke-keys"
Oct 30 13:19:11.806008 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:11.805961    4943 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6f0595d0-f37c-4067-b1e0-de986b38918b" containerName="revoke-keys"
Oct 30 13:19:11.806125 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:11.806114    4943 memory_manager.go:365] "RemoveStaleState removing state" podUID="6f0595d0-f37c-4067-b1e0-de986b38918b" containerName="revoke-keys"
Oct 30 13:19:11.810352 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:11.810331    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-acd1d13c-sbxnv"
Oct 30 13:19:11.834611 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:11.834579    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-acd1d13c-sbxnv"]
Oct 30 13:19:11.837097 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:11.837067    4943 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-p9z57\""
Oct 30 13:19:11.837670 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:11.837162    4943 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 13:19:11.903439 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:11.903402    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/0786b50b-b6e4-4788-bcd0-6fc3463e4d72-maas-api-xks-ca\") pod \"maas-api-revoke-sub-acd1d13c-sbxnv\" (UID: \"0786b50b-b6e4-4788-bcd0-6fc3463e4d72\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-acd1d13c-sbxnv"
Oct 30 13:19:11.903584 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:11.903461    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pw2zp\" (UniqueName: \"kubernetes.io/projected/0786b50b-b6e4-4788-bcd0-6fc3463e4d72-kube-api-access-pw2zp\") pod \"maas-api-revoke-sub-acd1d13c-sbxnv\" (UID: \"0786b50b-b6e4-4788-bcd0-6fc3463e4d72\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-acd1d13c-sbxnv"
Oct 30 13:19:11.903584 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:11.903568    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/0786b50b-b6e4-4788-bcd0-6fc3463e4d72-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-acd1d13c-sbxnv\" (UID: \"0786b50b-b6e4-4788-bcd0-6fc3463e4d72\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-acd1d13c-sbxnv"
Oct 30 13:19:12.004803 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:12.004727    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/0786b50b-b6e4-4788-bcd0-6fc3463e4d72-maas-api-xks-ca\") pod \"maas-api-revoke-sub-acd1d13c-sbxnv\" (UID: \"0786b50b-b6e4-4788-bcd0-6fc3463e4d72\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-acd1d13c-sbxnv"
Oct 30 13:19:12.004803 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:12.004774    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pw2zp\" (UniqueName: \"kubernetes.io/projected/0786b50b-b6e4-4788-bcd0-6fc3463e4d72-kube-api-access-pw2zp\") pod \"maas-api-revoke-sub-acd1d13c-sbxnv\" (UID: \"0786b50b-b6e4-4788-bcd0-6fc3463e4d72\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-acd1d13c-sbxnv"
Oct 30 13:19:12.004995 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:12.004814    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/0786b50b-b6e4-4788-bcd0-6fc3463e4d72-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-acd1d13c-sbxnv\" (UID: \"0786b50b-b6e4-4788-bcd0-6fc3463e4d72\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-acd1d13c-sbxnv"
Oct 30 13:19:12.005530 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:12.005510    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/0786b50b-b6e4-4788-bcd0-6fc3463e4d72-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-acd1d13c-sbxnv\" (UID: \"0786b50b-b6e4-4788-bcd0-6fc3463e4d72\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-acd1d13c-sbxnv"
Oct 30 13:19:12.007032 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:12.007006    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/0786b50b-b6e4-4788-bcd0-6fc3463e4d72-maas-api-xks-ca\") pod \"maas-api-revoke-sub-acd1d13c-sbxnv\" (UID: \"0786b50b-b6e4-4788-bcd0-6fc3463e4d72\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-acd1d13c-sbxnv"
Oct 30 13:19:12.030539 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:12.030514    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pw2zp\" (UniqueName: \"kubernetes.io/projected/0786b50b-b6e4-4788-bcd0-6fc3463e4d72-kube-api-access-pw2zp\") pod \"maas-api-revoke-sub-acd1d13c-sbxnv\" (UID: \"0786b50b-b6e4-4788-bcd0-6fc3463e4d72\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-acd1d13c-sbxnv"
Oct 30 13:19:12.142704 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:12.142669    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-acd1d13c-sbxnv"
Oct 30 13:19:12.521773 ip-10-0-142-84 kubenswrapper[4943]: W1030 13:19:12.521713    4943 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0786b50b_b6e4_4788_bcd0_6fc3463e4d72.slice/crio-d65e87f6c1e3524e5287aea7785f042e494c88209ca9af186388b0a7c6795802 WatchSource:0}: Error finding container d65e87f6c1e3524e5287aea7785f042e494c88209ca9af186388b0a7c6795802: Status 404 returned error can't find the container with id d65e87f6c1e3524e5287aea7785f042e494c88209ca9af186388b0a7c6795802
Oct 30 13:19:12.522306 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:12.522277    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-acd1d13c-sbxnv"]
Oct 30 13:19:12.847344 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:12.847309    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-acd1d13c-sbxnv" event={"ID":"0786b50b-b6e4-4788-bcd0-6fc3463e4d72","Type":"ContainerStarted","Data":"be9685d42c019f2312e664f602fa3280d036aeca530b9ec2d548663634e8e347"}
Oct 30 13:19:12.847679 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:12.847371    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-acd1d13c-sbxnv" event={"ID":"0786b50b-b6e4-4788-bcd0-6fc3463e4d72","Type":"ContainerStarted","Data":"d65e87f6c1e3524e5287aea7785f042e494c88209ca9af186388b0a7c6795802"}
Oct 30 13:19:12.911788 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:12.911746    4943 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-acd1d13c-sbxnv" podStartSLOduration=1.9117319940000002 podStartE2EDuration="1.911731994s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:19:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:19:12.910229472 +0000 UTC m=+1328.698233203" watchObservedRunningTime="2026-10-30 13:19:12.911731994 +0000 UTC m=+1328.699735724"
Oct 30 13:19:13.923720 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:13.923681    4943 generic.go:356] "Generic (PLEG): container finished" podID="0786b50b-b6e4-4788-bcd0-6fc3463e4d72" containerID="be9685d42c019f2312e664f602fa3280d036aeca530b9ec2d548663634e8e347" exitCode=0
Oct 30 13:19:13.924510 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:13.924479    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-acd1d13c-sbxnv" event={"ID":"0786b50b-b6e4-4788-bcd0-6fc3463e4d72","Type":"ContainerDied","Data":"be9685d42c019f2312e664f602fa3280d036aeca530b9ec2d548663634e8e347"}
Oct 30 13:19:14.946011 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:14.945978    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-hdmrm"]
Oct 30 13:19:15.102211 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:15.102185    4943 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-acd1d13c-sbxnv"
Oct 30 13:19:15.180842 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:15.180809    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/0786b50b-b6e4-4788-bcd0-6fc3463e4d72-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/0786b50b-b6e4-4788-bcd0-6fc3463e4d72-maas-api-xks-ca\") pod \"0786b50b-b6e4-4788-bcd0-6fc3463e4d72\" (UID: \"0786b50b-b6e4-4788-bcd0-6fc3463e4d72\") "
Oct 30 13:19:15.181051 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:15.180975    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0786b50b-b6e4-4788-bcd0-6fc3463e4d72-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/0786b50b-b6e4-4788-bcd0-6fc3463e4d72-maas-api-openshift-ca\") pod \"0786b50b-b6e4-4788-bcd0-6fc3463e4d72\" (UID: \"0786b50b-b6e4-4788-bcd0-6fc3463e4d72\") "
Oct 30 13:19:15.181051 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:15.181045    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0786b50b-b6e4-4788-bcd0-6fc3463e4d72-kube-api-access-pw2zp\" (UniqueName: \"kubernetes.io/projected/0786b50b-b6e4-4788-bcd0-6fc3463e4d72-kube-api-access-pw2zp\") pod \"0786b50b-b6e4-4788-bcd0-6fc3463e4d72\" (UID: \"0786b50b-b6e4-4788-bcd0-6fc3463e4d72\") "
Oct 30 13:19:15.181419 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:15.181390    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0786b50b-b6e4-4788-bcd0-6fc3463e4d72-maas-api-openshift-ca" pod "0786b50b-b6e4-4788-bcd0-6fc3463e4d72" (UID: "0786b50b-b6e4-4788-bcd0-6fc3463e4d72"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 13:19:15.181685 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:15.181663    4943 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/0786b50b-b6e4-4788-bcd0-6fc3463e4d72-maas-api-openshift-ca\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:19:15.183039 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:15.182986    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0786b50b-b6e4-4788-bcd0-6fc3463e4d72-maas-api-xks-ca" pod "0786b50b-b6e4-4788-bcd0-6fc3463e4d72" (UID: "0786b50b-b6e4-4788-bcd0-6fc3463e4d72"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 13:19:15.183311 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:15.183284    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0786b50b-b6e4-4788-bcd0-6fc3463e4d72-kube-api-access-pw2zp" pod "0786b50b-b6e4-4788-bcd0-6fc3463e4d72" (UID: "0786b50b-b6e4-4788-bcd0-6fc3463e4d72"). InnerVolumeSpecName "kube-api-access-pw2zp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:19:15.283599 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:15.283566    4943 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pw2zp\" (UniqueName: \"kubernetes.io/projected/0786b50b-b6e4-4788-bcd0-6fc3463e4d72-kube-api-access-pw2zp\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:19:15.283599 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:15.283596    4943 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/0786b50b-b6e4-4788-bcd0-6fc3463e4d72-maas-api-xks-ca\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:19:15.941452 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:15.941423    4943 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-acd1d13c-sbxnv"
Oct 30 13:19:15.941630 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:15.941450    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-acd1d13c-sbxnv" event={"ID":"0786b50b-b6e4-4788-bcd0-6fc3463e4d72","Type":"ContainerDied","Data":"d65e87f6c1e3524e5287aea7785f042e494c88209ca9af186388b0a7c6795802"}
Oct 30 13:19:15.941630 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:15.941502    4943 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d65e87f6c1e3524e5287aea7785f042e494c88209ca9af186388b0a7c6795802"
Oct 30 13:19:16.400334 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:16.399527    4943 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w5-f8a55e/distinct-w5-f8a55e-kserve-894fbd994-s2vsp" podUID="30f54ef4-28c2-4bf2-9feb-e2b0f1232c9f" containerName="main" containerID="cri-o://cb30fffcfc39cbf2406712457dc1353b9638d2c587563f00c7849722720d4823" gracePeriod=45
Oct 30 13:19:16.935065 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:16.935033    4943 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-f8a55e/distinct-w5-f8a55e-kserve-894fbd994-s2vsp"
Oct 30 13:19:16.951816 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:16.951781    4943 generic.go:356] "Generic (PLEG): container finished" podID="30f54ef4-28c2-4bf2-9feb-e2b0f1232c9f" containerID="cb30fffcfc39cbf2406712457dc1353b9638d2c587563f00c7849722720d4823" exitCode=0
Oct 30 13:19:16.951944 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:16.951836    4943 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-f8a55e/distinct-w5-f8a55e-kserve-894fbd994-s2vsp"
Oct 30 13:19:16.952022 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:16.951995    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-f8a55e/distinct-w5-f8a55e-kserve-894fbd994-s2vsp" event={"ID":"30f54ef4-28c2-4bf2-9feb-e2b0f1232c9f","Type":"ContainerDied","Data":"cb30fffcfc39cbf2406712457dc1353b9638d2c587563f00c7849722720d4823"}
Oct 30 13:19:16.952083 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:16.952026    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-f8a55e/distinct-w5-f8a55e-kserve-894fbd994-s2vsp" event={"ID":"30f54ef4-28c2-4bf2-9feb-e2b0f1232c9f","Type":"ContainerDied","Data":"0ddf785c8878aff5cca3b379282d3f4f62c2c2216850a7fba74a2e1e739e21bc"}
Oct 30 13:19:16.952083 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:16.952042    4943 scope.go:122] "RemoveContainer" containerID="cb30fffcfc39cbf2406712457dc1353b9638d2c587563f00c7849722720d4823"
Oct 30 13:19:16.963923 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:16.963898    4943 scope.go:122] "RemoveContainer" containerID="cb30fffcfc39cbf2406712457dc1353b9638d2c587563f00c7849722720d4823"
Oct 30 13:19:16.964344 ip-10-0-142-84 kubenswrapper[4943]: E1030 13:19:16.964315    4943 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cb30fffcfc39cbf2406712457dc1353b9638d2c587563f00c7849722720d4823\": container with ID starting with cb30fffcfc39cbf2406712457dc1353b9638d2c587563f00c7849722720d4823 not found: ID does not exist" containerID="cb30fffcfc39cbf2406712457dc1353b9638d2c587563f00c7849722720d4823"
Oct 30 13:19:16.964459 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:16.964350    4943 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cb30fffcfc39cbf2406712457dc1353b9638d2c587563f00c7849722720d4823"} err="failed to get container status \"cb30fffcfc39cbf2406712457dc1353b9638d2c587563f00c7849722720d4823\": rpc error: code = NotFound desc = could not find container \"cb30fffcfc39cbf2406712457dc1353b9638d2c587563f00c7849722720d4823\": container with ID starting with cb30fffcfc39cbf2406712457dc1353b9638d2c587563f00c7849722720d4823 not found: ID does not exist"
Oct 30 13:19:17.039282 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:17.037841    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/30f54ef4-28c2-4bf2-9feb-e2b0f1232c9f-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/30f54ef4-28c2-4bf2-9feb-e2b0f1232c9f-tmp-dir\") pod \"30f54ef4-28c2-4bf2-9feb-e2b0f1232c9f\" (UID: \"30f54ef4-28c2-4bf2-9feb-e2b0f1232c9f\") "
Oct 30 13:19:17.039282 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:17.037900    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/30f54ef4-28c2-4bf2-9feb-e2b0f1232c9f-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/30f54ef4-28c2-4bf2-9feb-e2b0f1232c9f-model-cache\") pod \"30f54ef4-28c2-4bf2-9feb-e2b0f1232c9f\" (UID: \"30f54ef4-28c2-4bf2-9feb-e2b0f1232c9f\") "
Oct 30 13:19:17.039282 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:17.037979    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/30f54ef4-28c2-4bf2-9feb-e2b0f1232c9f-home\" (UniqueName: \"kubernetes.io/empty-dir/30f54ef4-28c2-4bf2-9feb-e2b0f1232c9f-home\") pod \"30f54ef4-28c2-4bf2-9feb-e2b0f1232c9f\" (UID: \"30f54ef4-28c2-4bf2-9feb-e2b0f1232c9f\") "
Oct 30 13:19:17.039282 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:17.038142    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/30f54ef4-28c2-4bf2-9feb-e2b0f1232c9f-tls-certs\" (UniqueName: \"kubernetes.io/secret/30f54ef4-28c2-4bf2-9feb-e2b0f1232c9f-tls-certs\") pod \"30f54ef4-28c2-4bf2-9feb-e2b0f1232c9f\" (UID: \"30f54ef4-28c2-4bf2-9feb-e2b0f1232c9f\") "
Oct 30 13:19:17.039282 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:17.038196    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/30f54ef4-28c2-4bf2-9feb-e2b0f1232c9f-kube-api-access-k68z7\" (UniqueName: \"kubernetes.io/projected/30f54ef4-28c2-4bf2-9feb-e2b0f1232c9f-kube-api-access-k68z7\") pod \"30f54ef4-28c2-4bf2-9feb-e2b0f1232c9f\" (UID: \"30f54ef4-28c2-4bf2-9feb-e2b0f1232c9f\") "
Oct 30 13:19:17.040432 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:17.039904    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/30f54ef4-28c2-4bf2-9feb-e2b0f1232c9f-tmp-dir" pod "30f54ef4-28c2-4bf2-9feb-e2b0f1232c9f" (UID: "30f54ef4-28c2-4bf2-9feb-e2b0f1232c9f"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:19:17.040432 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:17.039948    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/30f54ef4-28c2-4bf2-9feb-e2b0f1232c9f-home" pod "30f54ef4-28c2-4bf2-9feb-e2b0f1232c9f" (UID: "30f54ef4-28c2-4bf2-9feb-e2b0f1232c9f"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:19:17.045649 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:17.044667    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/30f54ef4-28c2-4bf2-9feb-e2b0f1232c9f-kube-api-access-k68z7" pod "30f54ef4-28c2-4bf2-9feb-e2b0f1232c9f" (UID: "30f54ef4-28c2-4bf2-9feb-e2b0f1232c9f"). InnerVolumeSpecName "kube-api-access-k68z7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:19:17.046359 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:17.046321    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/30f54ef4-28c2-4bf2-9feb-e2b0f1232c9f-tls-certs" pod "30f54ef4-28c2-4bf2-9feb-e2b0f1232c9f" (UID: "30f54ef4-28c2-4bf2-9feb-e2b0f1232c9f"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 13:19:17.054393 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:17.054366    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/30f54ef4-28c2-4bf2-9feb-e2b0f1232c9f-dshm\" (UniqueName: \"kubernetes.io/empty-dir/30f54ef4-28c2-4bf2-9feb-e2b0f1232c9f-dshm\") pod \"30f54ef4-28c2-4bf2-9feb-e2b0f1232c9f\" (UID: \"30f54ef4-28c2-4bf2-9feb-e2b0f1232c9f\") "
Oct 30 13:19:17.055650 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:17.055555    4943 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/30f54ef4-28c2-4bf2-9feb-e2b0f1232c9f-tls-certs\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:19:17.055650 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:17.055586    4943 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k68z7\" (UniqueName: \"kubernetes.io/projected/30f54ef4-28c2-4bf2-9feb-e2b0f1232c9f-kube-api-access-k68z7\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:19:17.055650 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:17.055599    4943 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/30f54ef4-28c2-4bf2-9feb-e2b0f1232c9f-tmp-dir\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:19:17.055650 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:17.055613    4943 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/30f54ef4-28c2-4bf2-9feb-e2b0f1232c9f-home\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:19:17.056374 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:17.056346    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/30f54ef4-28c2-4bf2-9feb-e2b0f1232c9f-model-cache" pod "30f54ef4-28c2-4bf2-9feb-e2b0f1232c9f" (UID: "30f54ef4-28c2-4bf2-9feb-e2b0f1232c9f"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:19:17.056898 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:17.056874    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/30f54ef4-28c2-4bf2-9feb-e2b0f1232c9f-dshm" pod "30f54ef4-28c2-4bf2-9feb-e2b0f1232c9f" (UID: "30f54ef4-28c2-4bf2-9feb-e2b0f1232c9f"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:19:17.159521 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:17.158899    4943 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/30f54ef4-28c2-4bf2-9feb-e2b0f1232c9f-dshm\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:19:17.159521 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:17.158935    4943 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/30f54ef4-28c2-4bf2-9feb-e2b0f1232c9f-model-cache\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:19:17.435286 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:17.423478    4943 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w5-f8a55e/distinct-w5-f8a55e-kserve-894fbd994-s2vsp"]
Oct 30 13:19:17.459283 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:17.458604    4943 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w5-f8a55e/distinct-w5-f8a55e-kserve-894fbd994-s2vsp"]
Oct 30 13:19:17.893284 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:17.878352    4943 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-f8a55e/distinct-w5-f8a55e-kserve-894fbd994-s2vsp" podUID="30f54ef4-28c2-4bf2-9feb-e2b0f1232c9f" containerName="main" probeResult="failure" output="Get \"https://10.132.0.92:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:19:18.128282 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:18.127309    4943 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-f8a55e-697b9bfd75-pcr7h" podUID="93484267-ce16-4ae2-b52b-e46fae5e0121" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 13:19:18.356687 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:18.356652    4943 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="30f54ef4-28c2-4bf2-9feb-e2b0f1232c9f" path="/var/lib/kubelet/pods/30f54ef4-28c2-4bf2-9feb-e2b0f1232c9f/volumes"
Oct 30 13:19:18.793456 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:18.793414    4943 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-031737-rd656"]
Oct 30 13:19:18.794286 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:18.794245    4943 cpu_manager.go:414] "containerMap: removing container" podUID="30f54ef4-28c2-4bf2-9feb-e2b0f1232c9f" containerName="main"
Oct 30 13:19:18.794286 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:18.794282    4943 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="30f54ef4-28c2-4bf2-9feb-e2b0f1232c9f" containerName="main"
Oct 30 13:19:18.794453 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:18.794311    4943 cpu_manager.go:414] "containerMap: removing container" podUID="0786b50b-b6e4-4788-bcd0-6fc3463e4d72" containerName="revoke-keys"
Oct 30 13:19:18.794453 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:18.794318    4943 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0786b50b-b6e4-4788-bcd0-6fc3463e4d72" containerName="revoke-keys"
Oct 30 13:19:18.794555 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:18.794514    4943 memory_manager.go:365] "RemoveStaleState removing state" podUID="30f54ef4-28c2-4bf2-9feb-e2b0f1232c9f" containerName="main"
Oct 30 13:19:18.794555 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:18.794531    4943 memory_manager.go:365] "RemoveStaleState removing state" podUID="0786b50b-b6e4-4788-bcd0-6fc3463e4d72" containerName="revoke-keys"
Oct 30 13:19:18.799766 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:18.799743    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-031737-rd656"
Oct 30 13:19:18.820112 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:18.820085    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-031737-rd656"]
Oct 30 13:19:18.828603 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:18.828580    4943 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 13:19:18.828743 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:18.828685    4943 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-p9z57\""
Oct 30 13:19:18.855294 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:18.855240    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/963b4f75-ac69-43a1-9925-35acd0c9de8a-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-worker-w0-031737-rd656\" (UID: \"963b4f75-ac69-43a1-9925-35acd0c9de8a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-031737-rd656"
Oct 30 13:19:18.855421 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:18.855399    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/963b4f75-ac69-43a1-9925-35acd0c9de8a-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-worker-w0-031737-rd656\" (UID: \"963b4f75-ac69-43a1-9925-35acd0c9de8a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-031737-rd656"
Oct 30 13:19:18.855489 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:18.855427    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-txdpr\" (UniqueName: \"kubernetes.io/projected/963b4f75-ac69-43a1-9925-35acd0c9de8a-kube-api-access-txdpr\") pod \"maas-api-revoke-keys-e2e-worker-w0-031737-rd656\" (UID: \"963b4f75-ac69-43a1-9925-35acd0c9de8a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-031737-rd656"
Oct 30 13:19:18.956690 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:18.956655    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/963b4f75-ac69-43a1-9925-35acd0c9de8a-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-worker-w0-031737-rd656\" (UID: \"963b4f75-ac69-43a1-9925-35acd0c9de8a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-031737-rd656"
Oct 30 13:19:18.956851 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:18.956696    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-txdpr\" (UniqueName: \"kubernetes.io/projected/963b4f75-ac69-43a1-9925-35acd0c9de8a-kube-api-access-txdpr\") pod \"maas-api-revoke-keys-e2e-worker-w0-031737-rd656\" (UID: \"963b4f75-ac69-43a1-9925-35acd0c9de8a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-031737-rd656"
Oct 30 13:19:18.956936 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:18.956874    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/963b4f75-ac69-43a1-9925-35acd0c9de8a-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-worker-w0-031737-rd656\" (UID: \"963b4f75-ac69-43a1-9925-35acd0c9de8a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-031737-rd656"
Oct 30 13:19:18.957432 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:18.957407    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/963b4f75-ac69-43a1-9925-35acd0c9de8a-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-worker-w0-031737-rd656\" (UID: \"963b4f75-ac69-43a1-9925-35acd0c9de8a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-031737-rd656"
Oct 30 13:19:18.959320 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:18.959297    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/963b4f75-ac69-43a1-9925-35acd0c9de8a-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-worker-w0-031737-rd656\" (UID: \"963b4f75-ac69-43a1-9925-35acd0c9de8a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-031737-rd656"
Oct 30 13:19:18.982190 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:18.982160    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-txdpr\" (UniqueName: \"kubernetes.io/projected/963b4f75-ac69-43a1-9925-35acd0c9de8a-kube-api-access-txdpr\") pod \"maas-api-revoke-keys-e2e-worker-w0-031737-rd656\" (UID: \"963b4f75-ac69-43a1-9925-35acd0c9de8a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-031737-rd656"
Oct 30 13:19:19.114160 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:19.114095    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-031737-rd656"
Oct 30 13:19:19.431465 ip-10-0-142-84 kubenswrapper[4943]: W1030 13:19:19.431428    4943 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod963b4f75_ac69_43a1_9925_35acd0c9de8a.slice/crio-684590b297ef15fc71bfce4d02ba76341bbe3bfe0bf0b7c600a113691afd79ae WatchSource:0}: Error finding container 684590b297ef15fc71bfce4d02ba76341bbe3bfe0bf0b7c600a113691afd79ae: Status 404 returned error can't find the container with id 684590b297ef15fc71bfce4d02ba76341bbe3bfe0bf0b7c600a113691afd79ae
Oct 30 13:19:19.432409 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:19.432384    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-031737-rd656"]
Oct 30 13:19:20.009016 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:20.008973    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-031737-rd656" event={"ID":"963b4f75-ac69-43a1-9925-35acd0c9de8a","Type":"ContainerStarted","Data":"a961027a259c994b5f534c18a8147c7b1bdde4e2ec7a8e5ce1b2409914390c73"}
Oct 30 13:19:20.009520 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:20.009042    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-031737-rd656" event={"ID":"963b4f75-ac69-43a1-9925-35acd0c9de8a","Type":"ContainerStarted","Data":"684590b297ef15fc71bfce4d02ba76341bbe3bfe0bf0b7c600a113691afd79ae"}
Oct 30 13:19:20.077825 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:20.077733    4943 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-031737-rd656" podStartSLOduration=2.077719055 podStartE2EDuration="2.077719055s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:19:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:19:20.075042921 +0000 UTC m=+1335.863046755" watchObservedRunningTime="2026-10-30 13:19:20.077719055 +0000 UTC m=+1335.865722784"
Oct 30 13:19:21.018626 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:21.018585    4943 generic.go:356] "Generic (PLEG): container finished" podID="963b4f75-ac69-43a1-9925-35acd0c9de8a" containerID="a961027a259c994b5f534c18a8147c7b1bdde4e2ec7a8e5ce1b2409914390c73" exitCode=0
Oct 30 13:19:21.019350 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:21.019024    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-031737-rd656" event={"ID":"963b4f75-ac69-43a1-9925-35acd0c9de8a","Type":"ContainerDied","Data":"a961027a259c994b5f534c18a8147c7b1bdde4e2ec7a8e5ce1b2409914390c73"}
Oct 30 13:19:22.358704 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:22.352404    4943 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-031737-rd656"
Oct 30 13:19:22.455180 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:22.455145    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/963b4f75-ac69-43a1-9925-35acd0c9de8a-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/963b4f75-ac69-43a1-9925-35acd0c9de8a-maas-api-openshift-ca\") pod \"963b4f75-ac69-43a1-9925-35acd0c9de8a\" (UID: \"963b4f75-ac69-43a1-9925-35acd0c9de8a\") "
Oct 30 13:19:22.455380 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:22.455231    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/963b4f75-ac69-43a1-9925-35acd0c9de8a-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/963b4f75-ac69-43a1-9925-35acd0c9de8a-maas-api-xks-ca\") pod \"963b4f75-ac69-43a1-9925-35acd0c9de8a\" (UID: \"963b4f75-ac69-43a1-9925-35acd0c9de8a\") "
Oct 30 13:19:22.455380 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:22.455368    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/963b4f75-ac69-43a1-9925-35acd0c9de8a-kube-api-access-txdpr\" (UniqueName: \"kubernetes.io/projected/963b4f75-ac69-43a1-9925-35acd0c9de8a-kube-api-access-txdpr\") pod \"963b4f75-ac69-43a1-9925-35acd0c9de8a\" (UID: \"963b4f75-ac69-43a1-9925-35acd0c9de8a\") "
Oct 30 13:19:22.456632 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:22.456600    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/963b4f75-ac69-43a1-9925-35acd0c9de8a-maas-api-openshift-ca" pod "963b4f75-ac69-43a1-9925-35acd0c9de8a" (UID: "963b4f75-ac69-43a1-9925-35acd0c9de8a"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 13:19:22.457600 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:22.457575    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/963b4f75-ac69-43a1-9925-35acd0c9de8a-maas-api-xks-ca" pod "963b4f75-ac69-43a1-9925-35acd0c9de8a" (UID: "963b4f75-ac69-43a1-9925-35acd0c9de8a"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 13:19:22.458743 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:22.458723    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/963b4f75-ac69-43a1-9925-35acd0c9de8a-kube-api-access-txdpr" pod "963b4f75-ac69-43a1-9925-35acd0c9de8a" (UID: "963b4f75-ac69-43a1-9925-35acd0c9de8a"). InnerVolumeSpecName "kube-api-access-txdpr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:19:22.557152 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:22.557126    4943 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/963b4f75-ac69-43a1-9925-35acd0c9de8a-maas-api-openshift-ca\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:19:22.557152 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:22.557153    4943 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/963b4f75-ac69-43a1-9925-35acd0c9de8a-maas-api-xks-ca\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:19:22.557341 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:22.557163    4943 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-txdpr\" (UniqueName: \"kubernetes.io/projected/963b4f75-ac69-43a1-9925-35acd0c9de8a-kube-api-access-txdpr\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:19:23.043430 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:23.043390    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-031737-rd656" event={"ID":"963b4f75-ac69-43a1-9925-35acd0c9de8a","Type":"ContainerDied","Data":"684590b297ef15fc71bfce4d02ba76341bbe3bfe0bf0b7c600a113691afd79ae"}
Oct 30 13:19:23.043624 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:23.043447    4943 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="684590b297ef15fc71bfce4d02ba76341bbe3bfe0bf0b7c600a113691afd79ae"
Oct 30 13:19:23.043624 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:23.043398    4943 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-031737-rd656"
Oct 30 13:19:23.624712 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:23.624677    4943 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-031737/premium-w0-031737-kserve-58d5857cdc-vfnz9"]
Oct 30 13:19:28.180918 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:28.168406    4943 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-f8a55e-697b9bfd75-pcr7h" podUID="93484267-ce16-4ae2-b52b-e46fae5e0121" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 13:19:29.012073 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:29.012038    4943 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-031737-57d4df6b68-xr5s2"]
Oct 30 13:19:31.444986 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:31.444947    4943 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w5-f8a55e-6955dcb987-7t6p5"]
Oct 30 13:19:31.445429 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:31.445199    4943 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-f8a55e-6955dcb987-7t6p5" podUID="36d9a0d7-fa1d-463f-a424-3d36606422f3" containerName="maas-api" containerID="cri-o://6950212af90232cd183c627313d12524717292cc72163ac7b5d223f5ffc095eb" gracePeriod=30
Oct 30 13:19:31.911733 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:31.911700    4943 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-f8a55e-6955dcb987-7t6p5"
Oct 30 13:19:32.054618 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:32.054486    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/36d9a0d7-fa1d-463f-a424-3d36606422f3-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/36d9a0d7-fa1d-463f-a424-3d36606422f3-maas-api-tls\") pod \"36d9a0d7-fa1d-463f-a424-3d36606422f3\" (UID: \"36d9a0d7-fa1d-463f-a424-3d36606422f3\") "
Oct 30 13:19:32.055132 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:32.055104    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/36d9a0d7-fa1d-463f-a424-3d36606422f3-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/36d9a0d7-fa1d-463f-a424-3d36606422f3-openshift-service-ca\") pod \"36d9a0d7-fa1d-463f-a424-3d36606422f3\" (UID: \"36d9a0d7-fa1d-463f-a424-3d36606422f3\") "
Oct 30 13:19:32.055309 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:32.055288    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/36d9a0d7-fa1d-463f-a424-3d36606422f3-kube-api-access-29jnk\" (UniqueName: \"kubernetes.io/projected/36d9a0d7-fa1d-463f-a424-3d36606422f3-kube-api-access-29jnk\") pod \"36d9a0d7-fa1d-463f-a424-3d36606422f3\" (UID: \"36d9a0d7-fa1d-463f-a424-3d36606422f3\") "
Oct 30 13:19:32.059336 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:32.055476    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/36d9a0d7-fa1d-463f-a424-3d36606422f3-metrics-certs\" (UniqueName: \"kubernetes.io/secret/36d9a0d7-fa1d-463f-a424-3d36606422f3-metrics-certs\") pod \"36d9a0d7-fa1d-463f-a424-3d36606422f3\" (UID: \"36d9a0d7-fa1d-463f-a424-3d36606422f3\") "
Oct 30 13:19:32.062970 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:32.062627    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/36d9a0d7-fa1d-463f-a424-3d36606422f3-openshift-service-ca" pod "36d9a0d7-fa1d-463f-a424-3d36606422f3" (UID: "36d9a0d7-fa1d-463f-a424-3d36606422f3"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 13:19:32.182647 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:32.166173    4943 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/36d9a0d7-fa1d-463f-a424-3d36606422f3-openshift-service-ca\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:19:32.233671 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:32.212915    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/36d9a0d7-fa1d-463f-a424-3d36606422f3-kube-api-access-29jnk" pod "36d9a0d7-fa1d-463f-a424-3d36606422f3" (UID: "36d9a0d7-fa1d-463f-a424-3d36606422f3"). InnerVolumeSpecName "kube-api-access-29jnk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:19:32.233671 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:32.217279    4943 generic.go:356] "Generic (PLEG): container finished" podID="93484267-ce16-4ae2-b52b-e46fae5e0121" containerID="6391329407a4754f2b839e56094f340331c02341f487db984c4ed9943402c1f3" exitCode=0
Oct 30 13:19:32.233671 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:32.217486    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-f8a55e-697b9bfd75-pcr7h" event={"ID":"93484267-ce16-4ae2-b52b-e46fae5e0121","Type":"ContainerDied","Data":"6391329407a4754f2b839e56094f340331c02341f487db984c4ed9943402c1f3"}
Oct 30 13:19:32.350895 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:32.262786    4943 generic.go:356] "Generic (PLEG): container finished" podID="36d9a0d7-fa1d-463f-a424-3d36606422f3" containerID="6950212af90232cd183c627313d12524717292cc72163ac7b5d223f5ffc095eb" exitCode=0
Oct 30 13:19:32.350895 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:32.262947    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-f8a55e-6955dcb987-7t6p5" event={"ID":"36d9a0d7-fa1d-463f-a424-3d36606422f3","Type":"ContainerDied","Data":"6950212af90232cd183c627313d12524717292cc72163ac7b5d223f5ffc095eb"}
Oct 30 13:19:32.350895 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:32.263008    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-f8a55e-6955dcb987-7t6p5" event={"ID":"36d9a0d7-fa1d-463f-a424-3d36606422f3","Type":"ContainerDied","Data":"a5d4c350469a958520dd4d3a8ea07aa9a1bfde26004c7c3c73a4c6459ea4ee03"}
Oct 30 13:19:32.350895 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:32.263039    4943 scope.go:122] "RemoveContainer" containerID="6950212af90232cd183c627313d12524717292cc72163ac7b5d223f5ffc095eb"
Oct 30 13:19:32.350895 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:32.263288    4943 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-f8a55e-6955dcb987-7t6p5"
Oct 30 13:19:32.350895 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:32.267969    4943 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-29jnk\" (UniqueName: \"kubernetes.io/projected/36d9a0d7-fa1d-463f-a424-3d36606422f3-kube-api-access-29jnk\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:19:32.350895 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:32.343200    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/36d9a0d7-fa1d-463f-a424-3d36606422f3-metrics-certs" pod "36d9a0d7-fa1d-463f-a424-3d36606422f3" (UID: "36d9a0d7-fa1d-463f-a424-3d36606422f3"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 13:19:32.350895 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:32.343296    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/36d9a0d7-fa1d-463f-a424-3d36606422f3-maas-api-tls" pod "36d9a0d7-fa1d-463f-a424-3d36606422f3" (UID: "36d9a0d7-fa1d-463f-a424-3d36606422f3"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 13:19:32.379873 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:32.377609    4943 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/36d9a0d7-fa1d-463f-a424-3d36606422f3-metrics-certs\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:19:32.379873 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:32.377646    4943 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/36d9a0d7-fa1d-463f-a424-3d36606422f3-maas-api-tls\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:19:32.408776 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:32.408456    4943 scope.go:122] "RemoveContainer" containerID="6950212af90232cd183c627313d12524717292cc72163ac7b5d223f5ffc095eb"
Oct 30 13:19:32.409138 ip-10-0-142-84 kubenswrapper[4943]: E1030 13:19:32.409026    4943 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6950212af90232cd183c627313d12524717292cc72163ac7b5d223f5ffc095eb\": container with ID starting with 6950212af90232cd183c627313d12524717292cc72163ac7b5d223f5ffc095eb not found: ID does not exist" containerID="6950212af90232cd183c627313d12524717292cc72163ac7b5d223f5ffc095eb"
Oct 30 13:19:32.409138 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:32.409071    4943 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6950212af90232cd183c627313d12524717292cc72163ac7b5d223f5ffc095eb"} err="failed to get container status \"6950212af90232cd183c627313d12524717292cc72163ac7b5d223f5ffc095eb\": rpc error: code = NotFound desc = could not find container \"6950212af90232cd183c627313d12524717292cc72163ac7b5d223f5ffc095eb\": container with ID starting with 6950212af90232cd183c627313d12524717292cc72163ac7b5d223f5ffc095eb not found: ID does not exist"
Oct 30 13:19:32.448399 ip-10-0-142-84 kubenswrapper[4943]: E1030 13:19:32.448369    4943 cadvisor_stats_provider.go:569] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod36d9a0d7_fa1d_463f_a424_3d36606422f3.slice\": RecentStats: unable to find data in memory cache]"
Oct 30 13:19:32.493290 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:32.493253    4943 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-f8a55e-697b9bfd75-pcr7h"
Oct 30 13:19:32.580469 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:32.580441    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/93484267-ce16-4ae2-b52b-e46fae5e0121-kube-api-access-hfqwq\" (UniqueName: \"kubernetes.io/projected/93484267-ce16-4ae2-b52b-e46fae5e0121-kube-api-access-hfqwq\") pod \"93484267-ce16-4ae2-b52b-e46fae5e0121\" (UID: \"93484267-ce16-4ae2-b52b-e46fae5e0121\") "
Oct 30 13:19:32.580686 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:32.580671    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/93484267-ce16-4ae2-b52b-e46fae5e0121-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/93484267-ce16-4ae2-b52b-e46fae5e0121-plugins-config-volume\") pod \"93484267-ce16-4ae2-b52b-e46fae5e0121\" (UID: \"93484267-ce16-4ae2-b52b-e46fae5e0121\") "
Oct 30 13:19:32.581088 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:32.581057    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/93484267-ce16-4ae2-b52b-e46fae5e0121-plugins-config-volume" pod "93484267-ce16-4ae2-b52b-e46fae5e0121" (UID: "93484267-ce16-4ae2-b52b-e46fae5e0121"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 13:19:32.581203 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:32.581143    4943 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/93484267-ce16-4ae2-b52b-e46fae5e0121-plugins-config-volume\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:19:32.582757 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:32.582728    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/93484267-ce16-4ae2-b52b-e46fae5e0121-kube-api-access-hfqwq" pod "93484267-ce16-4ae2-b52b-e46fae5e0121" (UID: "93484267-ce16-4ae2-b52b-e46fae5e0121"). InnerVolumeSpecName "kube-api-access-hfqwq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:19:32.615920 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:32.615843    4943 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w5-f8a55e-6955dcb987-7t6p5"]
Oct 30 13:19:32.620344 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:32.620320    4943 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w5-f8a55e-6955dcb987-7t6p5"]
Oct 30 13:19:32.682803 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:32.682773    4943 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hfqwq\" (UniqueName: \"kubernetes.io/projected/93484267-ce16-4ae2-b52b-e46fae5e0121-kube-api-access-hfqwq\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:19:33.274600 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:33.274567    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-f8a55e-697b9bfd75-pcr7h" event={"ID":"93484267-ce16-4ae2-b52b-e46fae5e0121","Type":"ContainerDied","Data":"6029f2375e42b5588a6e879f40e2ce3c92ca8aaccd9a202dc80cb173ca6e87e0"}
Oct 30 13:19:33.274792 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:33.274610    4943 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-f8a55e-697b9bfd75-pcr7h"
Oct 30 13:19:33.274792 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:33.274625    4943 scope.go:122] "RemoveContainer" containerID="6391329407a4754f2b839e56094f340331c02341f487db984c4ed9943402c1f3"
Oct 30 13:19:33.337988 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:33.337951    4943 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w5-f8a55e-697b9bfd75-pcr7h"]
Oct 30 13:19:33.351139 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:33.351111    4943 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w5-f8a55e-697b9bfd75-pcr7h"]
Oct 30 13:19:33.529975 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:33.529903    4943 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w1-05942d-zf6sc"]
Oct 30 13:19:33.530620 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:33.530601    4943 cpu_manager.go:414] "containerMap: removing container" podUID="963b4f75-ac69-43a1-9925-35acd0c9de8a" containerName="revoke-keys"
Oct 30 13:19:33.530620 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:33.530621    4943 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="963b4f75-ac69-43a1-9925-35acd0c9de8a" containerName="revoke-keys"
Oct 30 13:19:33.530722 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:33.530651    4943 cpu_manager.go:414] "containerMap: removing container" podUID="93484267-ce16-4ae2-b52b-e46fae5e0121" containerName="payload-pre-processing"
Oct 30 13:19:33.530722 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:33.530657    4943 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="93484267-ce16-4ae2-b52b-e46fae5e0121" containerName="payload-pre-processing"
Oct 30 13:19:33.530722 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:33.530673    4943 cpu_manager.go:414] "containerMap: removing container" podUID="36d9a0d7-fa1d-463f-a424-3d36606422f3" containerName="maas-api"
Oct 30 13:19:33.530722 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:33.530678    4943 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="36d9a0d7-fa1d-463f-a424-3d36606422f3" containerName="maas-api"
Oct 30 13:19:33.530844 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:33.530783    4943 memory_manager.go:365] "RemoveStaleState removing state" podUID="963b4f75-ac69-43a1-9925-35acd0c9de8a" containerName="revoke-keys"
Oct 30 13:19:33.530844 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:33.530794    4943 memory_manager.go:365] "RemoveStaleState removing state" podUID="93484267-ce16-4ae2-b52b-e46fae5e0121" containerName="payload-pre-processing"
Oct 30 13:19:33.530844 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:33.530802    4943 memory_manager.go:365] "RemoveStaleState removing state" podUID="36d9a0d7-fa1d-463f-a424-3d36606422f3" containerName="maas-api"
Oct 30 13:19:33.535763 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:33.535743    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w1-05942d-zf6sc"
Oct 30 13:19:33.555746 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:33.555727    4943 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 13:19:33.555907 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:33.555879    4943 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Oct 30 13:19:33.556021 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:33.555965    4943 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Oct 30 13:19:33.556092 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:33.556059    4943 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-p9z57\""
Oct 30 13:19:33.562368 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:33.562347    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w1-05942d-zf6sc"]
Oct 30 13:19:33.594926 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:33.594895    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/03266586-4be7-42d3-85e5-e607b3754354-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-worker-w1-05942d-zf6sc\" (UID: \"03266586-4be7-42d3-85e5-e607b3754354\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w1-05942d-zf6sc"
Oct 30 13:19:33.595078 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:33.595034    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vt742\" (UniqueName: \"kubernetes.io/projected/03266586-4be7-42d3-85e5-e607b3754354-kube-api-access-vt742\") pod \"maas-api-revoke-keys-e2e-worker-w1-05942d-zf6sc\" (UID: \"03266586-4be7-42d3-85e5-e607b3754354\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w1-05942d-zf6sc"
Oct 30 13:19:33.595161 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:33.595144    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/03266586-4be7-42d3-85e5-e607b3754354-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-worker-w1-05942d-zf6sc\" (UID: \"03266586-4be7-42d3-85e5-e607b3754354\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w1-05942d-zf6sc"
Oct 30 13:19:33.696332 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:33.696298    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/03266586-4be7-42d3-85e5-e607b3754354-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-worker-w1-05942d-zf6sc\" (UID: \"03266586-4be7-42d3-85e5-e607b3754354\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w1-05942d-zf6sc"
Oct 30 13:19:33.696477 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:33.696398    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vt742\" (UniqueName: \"kubernetes.io/projected/03266586-4be7-42d3-85e5-e607b3754354-kube-api-access-vt742\") pod \"maas-api-revoke-keys-e2e-worker-w1-05942d-zf6sc\" (UID: \"03266586-4be7-42d3-85e5-e607b3754354\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w1-05942d-zf6sc"
Oct 30 13:19:33.696681 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:33.696492    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/03266586-4be7-42d3-85e5-e607b3754354-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-worker-w1-05942d-zf6sc\" (UID: \"03266586-4be7-42d3-85e5-e607b3754354\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w1-05942d-zf6sc"
Oct 30 13:19:33.697155 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:33.697136    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/03266586-4be7-42d3-85e5-e607b3754354-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-worker-w1-05942d-zf6sc\" (UID: \"03266586-4be7-42d3-85e5-e607b3754354\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w1-05942d-zf6sc"
Oct 30 13:19:33.698772 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:33.698751    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/03266586-4be7-42d3-85e5-e607b3754354-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-worker-w1-05942d-zf6sc\" (UID: \"03266586-4be7-42d3-85e5-e607b3754354\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w1-05942d-zf6sc"
Oct 30 13:19:33.722193 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:33.722169    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vt742\" (UniqueName: \"kubernetes.io/projected/03266586-4be7-42d3-85e5-e607b3754354-kube-api-access-vt742\") pod \"maas-api-revoke-keys-e2e-worker-w1-05942d-zf6sc\" (UID: \"03266586-4be7-42d3-85e5-e607b3754354\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w1-05942d-zf6sc"
Oct 30 13:19:33.772724 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:33.772698    4943 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-06c73c5c-lzbxz"]
Oct 30 13:19:33.780949 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:33.780879    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-06c73c5c-lzbxz"
Oct 30 13:19:33.798130 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:33.798104    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-06c73c5c-lzbxz"]
Oct 30 13:19:33.849532 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:33.849504    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w1-05942d-zf6sc"
Oct 30 13:19:33.899381 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:33.899344    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/4ffb2c71-a66d-4f4f-8fdd-48707259c348-maas-api-xks-ca\") pod \"maas-api-revoke-sub-06c73c5c-lzbxz\" (UID: \"4ffb2c71-a66d-4f4f-8fdd-48707259c348\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-06c73c5c-lzbxz"
Oct 30 13:19:33.899569 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:33.899543    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jtlml\" (UniqueName: \"kubernetes.io/projected/4ffb2c71-a66d-4f4f-8fdd-48707259c348-kube-api-access-jtlml\") pod \"maas-api-revoke-sub-06c73c5c-lzbxz\" (UID: \"4ffb2c71-a66d-4f4f-8fdd-48707259c348\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-06c73c5c-lzbxz"
Oct 30 13:19:33.899684 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:33.899669    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/4ffb2c71-a66d-4f4f-8fdd-48707259c348-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-06c73c5c-lzbxz\" (UID: \"4ffb2c71-a66d-4f4f-8fdd-48707259c348\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-06c73c5c-lzbxz"
Oct 30 13:19:34.001500 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:34.001462    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jtlml\" (UniqueName: \"kubernetes.io/projected/4ffb2c71-a66d-4f4f-8fdd-48707259c348-kube-api-access-jtlml\") pod \"maas-api-revoke-sub-06c73c5c-lzbxz\" (UID: \"4ffb2c71-a66d-4f4f-8fdd-48707259c348\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-06c73c5c-lzbxz"
Oct 30 13:19:34.001674 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:34.001540    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/4ffb2c71-a66d-4f4f-8fdd-48707259c348-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-06c73c5c-lzbxz\" (UID: \"4ffb2c71-a66d-4f4f-8fdd-48707259c348\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-06c73c5c-lzbxz"
Oct 30 13:19:34.001674 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:34.001638    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/4ffb2c71-a66d-4f4f-8fdd-48707259c348-maas-api-xks-ca\") pod \"maas-api-revoke-sub-06c73c5c-lzbxz\" (UID: \"4ffb2c71-a66d-4f4f-8fdd-48707259c348\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-06c73c5c-lzbxz"
Oct 30 13:19:34.002427 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:34.002397    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/4ffb2c71-a66d-4f4f-8fdd-48707259c348-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-06c73c5c-lzbxz\" (UID: \"4ffb2c71-a66d-4f4f-8fdd-48707259c348\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-06c73c5c-lzbxz"
Oct 30 13:19:34.004020 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:34.003999    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/4ffb2c71-a66d-4f4f-8fdd-48707259c348-maas-api-xks-ca\") pod \"maas-api-revoke-sub-06c73c5c-lzbxz\" (UID: \"4ffb2c71-a66d-4f4f-8fdd-48707259c348\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-06c73c5c-lzbxz"
Oct 30 13:19:34.013323 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:34.013297    4943 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-w0-031737-57d4df6b68-xr5s2" podUID="9fc1504d-012b-4afd-bebf-7ac106d6990e" containerName="payload-processing" containerID="cri-o://2f655124e975981258127d10703872d21c94f5f1dedd7ae7c29ca637f5fbbd4b" gracePeriod=25
Oct 30 13:19:34.026801 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:34.026779    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jtlml\" (UniqueName: \"kubernetes.io/projected/4ffb2c71-a66d-4f4f-8fdd-48707259c348-kube-api-access-jtlml\") pod \"maas-api-revoke-sub-06c73c5c-lzbxz\" (UID: \"4ffb2c71-a66d-4f4f-8fdd-48707259c348\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-06c73c5c-lzbxz"
Oct 30 13:19:34.098413 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:34.098355    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-06c73c5c-lzbxz"
Oct 30 13:19:34.162988 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:34.162915    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w1-05942d-zf6sc"]
Oct 30 13:19:34.321851 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:34.321815    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w1-05942d-zf6sc" event={"ID":"03266586-4be7-42d3-85e5-e607b3754354","Type":"ContainerStarted","Data":"9d9cd685b14728a82305cc031a28376b412b94c2fb6d14b24646dcd71ef23206"}
Oct 30 13:19:34.353087 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:34.353008    4943 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="36d9a0d7-fa1d-463f-a424-3d36606422f3" path="/var/lib/kubelet/pods/36d9a0d7-fa1d-463f-a424-3d36606422f3/volumes"
Oct 30 13:19:34.353508 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:34.353489    4943 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="93484267-ce16-4ae2-b52b-e46fae5e0121" path="/var/lib/kubelet/pods/93484267-ce16-4ae2-b52b-e46fae5e0121/volumes"
Oct 30 13:19:34.425507 ip-10-0-142-84 kubenswrapper[4943]: W1030 13:19:34.425471    4943 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4ffb2c71_a66d_4f4f_8fdd_48707259c348.slice/crio-e7fa40ecd14f1359a2dbff9552f5bd692a1359430ca821d6fa94a9b21d0ee1a7 WatchSource:0}: Error finding container e7fa40ecd14f1359a2dbff9552f5bd692a1359430ca821d6fa94a9b21d0ee1a7: Status 404 returned error can't find the container with id e7fa40ecd14f1359a2dbff9552f5bd692a1359430ca821d6fa94a9b21d0ee1a7
Oct 30 13:19:34.426184 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:34.426081    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-06c73c5c-lzbxz"]
Oct 30 13:19:35.333290 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:35.333226    4943 generic.go:356] "Generic (PLEG): container finished" podID="4ffb2c71-a66d-4f4f-8fdd-48707259c348" containerID="09f9ec76060fad5a818fee4d217d80149fdc85091c1a08150d9c387862920924" exitCode=0
Oct 30 13:19:35.333668 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:35.333290    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-06c73c5c-lzbxz" event={"ID":"4ffb2c71-a66d-4f4f-8fdd-48707259c348","Type":"ContainerDied","Data":"09f9ec76060fad5a818fee4d217d80149fdc85091c1a08150d9c387862920924"}
Oct 30 13:19:35.333668 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:35.333346    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-06c73c5c-lzbxz" event={"ID":"4ffb2c71-a66d-4f4f-8fdd-48707259c348","Type":"ContainerStarted","Data":"e7fa40ecd14f1359a2dbff9552f5bd692a1359430ca821d6fa94a9b21d0ee1a7"}
Oct 30 13:19:35.335150 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:35.335128    4943 generic.go:356] "Generic (PLEG): container finished" podID="03266586-4be7-42d3-85e5-e607b3754354" containerID="0bac164e5acfb9089fa1f0bd80606e742e958b61beca4f3118f1f3af2545b0c8" exitCode=0
Oct 30 13:19:35.335331 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:35.335170    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w1-05942d-zf6sc" event={"ID":"03266586-4be7-42d3-85e5-e607b3754354","Type":"ContainerDied","Data":"0bac164e5acfb9089fa1f0bd80606e742e958b61beca4f3118f1f3af2545b0c8"}
Oct 30 13:19:35.958029 ip-10-0-142-84 kubenswrapper[4943]: E1030 13:19:35.957993    4943 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-knqz6" containerName="extractor"
Oct 30 13:19:36.513937 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:36.513908    4943 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w1-05942d-zf6sc"
Oct 30 13:19:36.550389 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:36.550361    4943 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-06c73c5c-lzbxz"
Oct 30 13:19:36.656674 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:36.656586    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/03266586-4be7-42d3-85e5-e607b3754354-kube-api-access-vt742\" (UniqueName: \"kubernetes.io/projected/03266586-4be7-42d3-85e5-e607b3754354-kube-api-access-vt742\") pod \"03266586-4be7-42d3-85e5-e607b3754354\" (UID: \"03266586-4be7-42d3-85e5-e607b3754354\") "
Oct 30 13:19:36.656827 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:36.656735    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/4ffb2c71-a66d-4f4f-8fdd-48707259c348-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/4ffb2c71-a66d-4f4f-8fdd-48707259c348-maas-api-openshift-ca\") pod \"4ffb2c71-a66d-4f4f-8fdd-48707259c348\" (UID: \"4ffb2c71-a66d-4f4f-8fdd-48707259c348\") "
Oct 30 13:19:36.656827 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:36.656786    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/4ffb2c71-a66d-4f4f-8fdd-48707259c348-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/4ffb2c71-a66d-4f4f-8fdd-48707259c348-maas-api-xks-ca\") pod \"4ffb2c71-a66d-4f4f-8fdd-48707259c348\" (UID: \"4ffb2c71-a66d-4f4f-8fdd-48707259c348\") "
Oct 30 13:19:36.656961 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:36.656837    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/03266586-4be7-42d3-85e5-e607b3754354-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/03266586-4be7-42d3-85e5-e607b3754354-maas-api-openshift-ca\") pod \"03266586-4be7-42d3-85e5-e607b3754354\" (UID: \"03266586-4be7-42d3-85e5-e607b3754354\") "
Oct 30 13:19:36.657161 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:36.657097    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/03266586-4be7-42d3-85e5-e607b3754354-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/03266586-4be7-42d3-85e5-e607b3754354-maas-api-xks-ca\") pod \"03266586-4be7-42d3-85e5-e607b3754354\" (UID: \"03266586-4be7-42d3-85e5-e607b3754354\") "
Oct 30 13:19:36.657285 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:36.657153    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4ffb2c71-a66d-4f4f-8fdd-48707259c348-maas-api-openshift-ca" pod "4ffb2c71-a66d-4f4f-8fdd-48707259c348" (UID: "4ffb2c71-a66d-4f4f-8fdd-48707259c348"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 13:19:36.657285 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:36.657179    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4ffb2c71-a66d-4f4f-8fdd-48707259c348-kube-api-access-jtlml\" (UniqueName: \"kubernetes.io/projected/4ffb2c71-a66d-4f4f-8fdd-48707259c348-kube-api-access-jtlml\") pod \"4ffb2c71-a66d-4f4f-8fdd-48707259c348\" (UID: \"4ffb2c71-a66d-4f4f-8fdd-48707259c348\") "
Oct 30 13:19:36.657463 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:36.657277    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/03266586-4be7-42d3-85e5-e607b3754354-maas-api-openshift-ca" pod "03266586-4be7-42d3-85e5-e607b3754354" (UID: "03266586-4be7-42d3-85e5-e607b3754354"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 13:19:36.657803 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:36.657784    4943 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/4ffb2c71-a66d-4f4f-8fdd-48707259c348-maas-api-openshift-ca\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:19:36.657803 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:36.657804    4943 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/03266586-4be7-42d3-85e5-e607b3754354-maas-api-openshift-ca\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:19:36.659092 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:36.659066    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/03266586-4be7-42d3-85e5-e607b3754354-maas-api-xks-ca" pod "03266586-4be7-42d3-85e5-e607b3754354" (UID: "03266586-4be7-42d3-85e5-e607b3754354"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 13:19:36.659209 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:36.659192    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4ffb2c71-a66d-4f4f-8fdd-48707259c348-kube-api-access-jtlml" pod "4ffb2c71-a66d-4f4f-8fdd-48707259c348" (UID: "4ffb2c71-a66d-4f4f-8fdd-48707259c348"). InnerVolumeSpecName "kube-api-access-jtlml". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:19:36.659313 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:36.659295    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4ffb2c71-a66d-4f4f-8fdd-48707259c348-maas-api-xks-ca" pod "4ffb2c71-a66d-4f4f-8fdd-48707259c348" (UID: "4ffb2c71-a66d-4f4f-8fdd-48707259c348"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 13:19:36.659373 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:36.659347    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/03266586-4be7-42d3-85e5-e607b3754354-kube-api-access-vt742" pod "03266586-4be7-42d3-85e5-e607b3754354" (UID: "03266586-4be7-42d3-85e5-e607b3754354"). InnerVolumeSpecName "kube-api-access-vt742". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:19:36.758986 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:36.758952    4943 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/03266586-4be7-42d3-85e5-e607b3754354-maas-api-xks-ca\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:19:36.758986 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:36.758985    4943 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jtlml\" (UniqueName: \"kubernetes.io/projected/4ffb2c71-a66d-4f4f-8fdd-48707259c348-kube-api-access-jtlml\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:19:36.759149 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:36.759000    4943 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vt742\" (UniqueName: \"kubernetes.io/projected/03266586-4be7-42d3-85e5-e607b3754354-kube-api-access-vt742\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:19:36.759149 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:36.759013    4943 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/4ffb2c71-a66d-4f4f-8fdd-48707259c348-maas-api-xks-ca\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:19:37.352256 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:37.352218    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w1-05942d-zf6sc" event={"ID":"03266586-4be7-42d3-85e5-e607b3754354","Type":"ContainerDied","Data":"9d9cd685b14728a82305cc031a28376b412b94c2fb6d14b24646dcd71ef23206"}
Oct 30 13:19:37.352444 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:37.352292    4943 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9d9cd685b14728a82305cc031a28376b412b94c2fb6d14b24646dcd71ef23206"
Oct 30 13:19:37.352444 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:37.352361    4943 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w1-05942d-zf6sc"
Oct 30 13:19:37.357548 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:37.357496    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-06c73c5c-lzbxz" event={"ID":"4ffb2c71-a66d-4f4f-8fdd-48707259c348","Type":"ContainerDied","Data":"e7fa40ecd14f1359a2dbff9552f5bd692a1359430ca821d6fa94a9b21d0ee1a7"}
Oct 30 13:19:37.357691 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:37.357624    4943 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e7fa40ecd14f1359a2dbff9552f5bd692a1359430ca821d6fa94a9b21d0ee1a7"
Oct 30 13:19:37.357749 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:37.357720    4943 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-06c73c5c-lzbxz"
Oct 30 13:19:38.459282 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:38.458911    4943 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w1-05942d/distinct-2-w1-05942d-kserve-855c7f7c5f-pwt79"]
Oct 30 13:19:38.516297 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:38.500810    4943 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w1-05942d/distinct-w1-05942d-kserve-5fb44dc664-mcbkc"]
Oct 30 13:19:38.673284 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:38.668585    4943 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-031737/premium-w0-031737-kserve-58d5857cdc-vfnz9" podUID="79b1a638-b019-4c89-b850-01699ecac9ba" containerName="main" containerID="cri-o://8866cbc1f1139e188db329cb54c34b31771bd730a2a1ba37f03bd113a853fbcf" gracePeriod=45
Oct 30 13:19:38.724339 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:38.715473    4943 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-031737/premium-w0-031737-kserve-58d5857cdc-vfnz9" podUID="79b1a638-b019-4c89-b850-01699ecac9ba" containerName="main" probeResult="failure" output="Get \"https://10.132.0.70:8000/ready\": read tcp 10.132.0.2:52292->10.132.0.70:8000: read: connection reset by peer"
Oct 30 13:19:39.574585 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:39.574307    4943 generic.go:356] "Generic (PLEG): container finished" podID="79b1a638-b019-4c89-b850-01699ecac9ba" containerID="8866cbc1f1139e188db329cb54c34b31771bd730a2a1ba37f03bd113a853fbcf" exitCode=0
Oct 30 13:19:39.574585 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:39.574521    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-031737/premium-w0-031737-kserve-58d5857cdc-vfnz9" event={"ID":"79b1a638-b019-4c89-b850-01699ecac9ba","Type":"ContainerDied","Data":"8866cbc1f1139e188db329cb54c34b31771bd730a2a1ba37f03bd113a853fbcf"}
Oct 30 13:19:39.680749 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:39.680693    4943 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-031737/premium-w0-031737-kserve-58d5857cdc-vfnz9" podUID="79b1a638-b019-4c89-b850-01699ecac9ba" containerName="main" probeResult="failure" output="Get \"https://10.132.0.70:8000/ready\": dial tcp 10.132.0.70:8000: connect: connection refused"
Oct 30 13:19:39.686377 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:39.686352    4943 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-031737/premium-w0-031737-kserve-58d5857cdc-vfnz9"
Oct 30 13:19:40.148638 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:40.148592    4943 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-d0a18c/distinct-w3-d0a18c-kserve-86479f6bbf-5zcqq" podUID="63dace4c-7832-47dd-84ab-2c91483a5563" containerName="main" probeResult="failure" output="Get \"https://10.132.0.89:8000/ready\": context deadline exceeded"
Oct 30 13:19:40.149479 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:40.149439    4943 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-05942d/distinct-w1-05942d-kserve-5fb44dc664-mcbkc" podUID="d08eb359-cf9e-422c-a3a7-35fd173ab7b6" containerName="main" probeResult="failure" output="Get \"https://10.132.0.91:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:19:40.583287 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:40.557310    4943 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-ab3176/simulator-w6-ab3176-kserve-6d9b64549f-dc8xz" podUID="da306792-4f80-4df4-aa44-82991ab856e4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.69:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:19:40.583287 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:40.568156    4943 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-d0a18c/distinct-2-w3-d0a18c-kserve-85f8ff4846-rc44r" podUID="d65e47d2-4461-406a-9b71-bb3fac4262f0" containerName="main" probeResult="failure" output="Get \"https://10.132.0.93:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:19:41.163556 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:41.149723    4943 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-05942d/distinct-w1-05942d-kserve-5fb44dc664-mcbkc" podUID="d08eb359-cf9e-422c-a3a7-35fd173ab7b6" containerName="main" probeResult="failure" output="Get \"https://10.132.0.91:8000/ready\": context deadline exceeded"
Oct 30 13:19:41.163556 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:41.149655    4943 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-d0a18c/distinct-w3-d0a18c-kserve-86479f6bbf-5zcqq" podUID="63dace4c-7832-47dd-84ab-2c91483a5563" containerName="main" probeResult="failure" output="Get \"https://10.132.0.89:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:19:41.163556 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:41.152234    4943 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-05942d/distinct-w1-05942d-kserve-5fb44dc664-mcbkc"
Oct 30 13:19:41.220444 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:41.213699    4943 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-d0a18c/distinct-w3-d0a18c-kserve-86479f6bbf-5zcqq"
Oct 30 13:19:41.241284 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:41.221709    4943 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-031737/premium-w0-031737-kserve-58d5857cdc-vfnz9"
Oct 30 13:19:41.294737 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:41.294645    4943 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-05942d/distinct-2-w1-05942d-kserve-855c7f7c5f-pwt79" podUID="3dac65dc-7a67-4fdb-840a-765235aaa5bf" containerName="main" probeResult="failure" output="Get \"https://10.132.0.95:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:19:41.411033 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:41.411001    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/79b1a638-b019-4c89-b850-01699ecac9ba-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/79b1a638-b019-4c89-b850-01699ecac9ba-model-cache\") pod \"79b1a638-b019-4c89-b850-01699ecac9ba\" (UID: \"79b1a638-b019-4c89-b850-01699ecac9ba\") "
Oct 30 13:19:41.413526 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:41.413489    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/79b1a638-b019-4c89-b850-01699ecac9ba-dshm\" (UniqueName: \"kubernetes.io/empty-dir/79b1a638-b019-4c89-b850-01699ecac9ba-dshm\") pod \"79b1a638-b019-4c89-b850-01699ecac9ba\" (UID: \"79b1a638-b019-4c89-b850-01699ecac9ba\") "
Oct 30 13:19:41.421460 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:41.413378    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/79b1a638-b019-4c89-b850-01699ecac9ba-model-cache" pod "79b1a638-b019-4c89-b850-01699ecac9ba" (UID: "79b1a638-b019-4c89-b850-01699ecac9ba"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:19:41.421796 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:41.414415    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/79b1a638-b019-4c89-b850-01699ecac9ba-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/79b1a638-b019-4c89-b850-01699ecac9ba-tmp-dir\") pod \"79b1a638-b019-4c89-b850-01699ecac9ba\" (UID: \"79b1a638-b019-4c89-b850-01699ecac9ba\") "
Oct 30 13:19:41.422442 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:41.422414    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/79b1a638-b019-4c89-b850-01699ecac9ba-home\" (UniqueName: \"kubernetes.io/empty-dir/79b1a638-b019-4c89-b850-01699ecac9ba-home\") pod \"79b1a638-b019-4c89-b850-01699ecac9ba\" (UID: \"79b1a638-b019-4c89-b850-01699ecac9ba\") "
Oct 30 13:19:41.428944 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:41.428829    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/79b1a638-b019-4c89-b850-01699ecac9ba-kube-api-access-q62fd\" (UniqueName: \"kubernetes.io/projected/79b1a638-b019-4c89-b850-01699ecac9ba-kube-api-access-q62fd\") pod \"79b1a638-b019-4c89-b850-01699ecac9ba\" (UID: \"79b1a638-b019-4c89-b850-01699ecac9ba\") "
Oct 30 13:19:41.429242 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:41.429130    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/79b1a638-b019-4c89-b850-01699ecac9ba-tls-certs\" (UniqueName: \"kubernetes.io/secret/79b1a638-b019-4c89-b850-01699ecac9ba-tls-certs\") pod \"79b1a638-b019-4c89-b850-01699ecac9ba\" (UID: \"79b1a638-b019-4c89-b850-01699ecac9ba\") "
Oct 30 13:19:41.436476 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:41.428572    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/79b1a638-b019-4c89-b850-01699ecac9ba-home" pod "79b1a638-b019-4c89-b850-01699ecac9ba" (UID: "79b1a638-b019-4c89-b850-01699ecac9ba"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:19:41.436633 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:41.428717    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/79b1a638-b019-4c89-b850-01699ecac9ba-tmp-dir" pod "79b1a638-b019-4c89-b850-01699ecac9ba" (UID: "79b1a638-b019-4c89-b850-01699ecac9ba"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:19:41.479279 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:41.479238    4943 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/79b1a638-b019-4c89-b850-01699ecac9ba-model-cache\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:19:41.479960 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:41.479921    4943 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/79b1a638-b019-4c89-b850-01699ecac9ba-tmp-dir\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:19:41.480465 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:41.480448    4943 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/79b1a638-b019-4c89-b850-01699ecac9ba-home\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:19:41.568343 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:41.560365    4943 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-ab3176/simulator-w6-ab3176-kserve-6d9b64549f-dc8xz" podUID="da306792-4f80-4df4-aa44-82991ab856e4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.69:8000/ready\": context deadline exceeded"
Oct 30 13:19:41.603569 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:41.568751    4943 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-ab3176/simulator-w6-ab3176-kserve-6d9b64549f-dc8xz"
Oct 30 13:19:41.603569 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:41.571705    4943 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-d0a18c/distinct-2-w3-d0a18c-kserve-85f8ff4846-rc44r" podUID="d65e47d2-4461-406a-9b71-bb3fac4262f0" containerName="main" probeResult="failure" output="Get \"https://10.132.0.93:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:19:41.603569 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:41.572079    4943 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-d0a18c/distinct-2-w3-d0a18c-kserve-85f8ff4846-rc44r"
Oct 30 13:19:41.626414 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:41.623429    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/79b1a638-b019-4c89-b850-01699ecac9ba-kube-api-access-q62fd" pod "79b1a638-b019-4c89-b850-01699ecac9ba" (UID: "79b1a638-b019-4c89-b850-01699ecac9ba"). InnerVolumeSpecName "kube-api-access-q62fd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:19:41.652301 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:41.634014    4943 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q62fd\" (UniqueName: \"kubernetes.io/projected/79b1a638-b019-4c89-b850-01699ecac9ba-kube-api-access-q62fd\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:19:41.691283 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:41.674357    4943 patch_prober.go:29] interesting pod/e2e-worker-w2-f76c09-openshift-default-65d5845b67-kpvbd container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.85:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 13:19:41.691283 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:41.674430    4943 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-kpvbd" podUID="f0e82596-0383-4ec9-a856-a6234e9da808" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.85:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:19:41.691283 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:41.677395    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/79b1a638-b019-4c89-b850-01699ecac9ba-tls-certs" pod "79b1a638-b019-4c89-b850-01699ecac9ba" (UID: "79b1a638-b019-4c89-b850-01699ecac9ba"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 13:19:41.748287 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:41.739363    4943 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-031737/premium-w0-031737-kserve-58d5857cdc-vfnz9" podUID="79b1a638-b019-4c89-b850-01699ecac9ba" containerName="main" probeResult="failure" output="Get \"https://10.132.0.70:8000/ready\": context deadline exceeded"
Oct 30 13:19:41.748287 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:41.742954    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/79b1a638-b019-4c89-b850-01699ecac9ba-dshm" pod "79b1a638-b019-4c89-b850-01699ecac9ba" (UID: "79b1a638-b019-4c89-b850-01699ecac9ba"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:19:41.809285 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:41.783722    4943 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/79b1a638-b019-4c89-b850-01699ecac9ba-dshm\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:19:41.817286 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:41.809561    4943 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/79b1a638-b019-4c89-b850-01699ecac9ba-tls-certs\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:19:42.183287 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:42.158709    4943 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-d0a18c/distinct-w3-d0a18c-kserve-86479f6bbf-5zcqq" podUID="63dace4c-7832-47dd-84ab-2c91483a5563" containerName="main" probeResult="failure" output="Get \"https://10.132.0.89:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:19:42.458340 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:42.208903    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-031737/premium-w0-031737-kserve-58d5857cdc-vfnz9" event={"ID":"79b1a638-b019-4c89-b850-01699ecac9ba","Type":"ContainerDied","Data":"f1bd293d49314f1ab9a0ae336edf53625ec8316216f85a1e76b8d3ebff85780e"}
Oct 30 13:19:42.507332 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:42.209006    4943 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-031737/premium-w0-031737-kserve-58d5857cdc-vfnz9"
Oct 30 13:19:42.896007 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:42.288351    4943 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w3-d0a18c/distinct-w3-d0a18c-kserve-86479f6bbf-5zcqq" podUID="63dace4c-7832-47dd-84ab-2c91483a5563" containerName="main" probeResult="failure" output="Get \"https://10.132.0.89:8000/health\": context deadline exceeded"
Oct 30 13:19:42.896007 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:42.310471    4943 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-05942d/distinct-2-w1-05942d-kserve-855c7f7c5f-pwt79" podUID="3dac65dc-7a67-4fdb-840a-765235aaa5bf" containerName="main" probeResult="failure" output="Get \"https://10.132.0.95:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:19:42.896007 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:42.411942    4943 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-05942d/distinct-w1-05942d-kserve-5fb44dc664-mcbkc" podUID="d08eb359-cf9e-422c-a3a7-35fd173ab7b6" containerName="main" probeResult="failure" output="Get \"https://10.132.0.91:8000/ready\": context deadline exceeded"
Oct 30 13:19:42.896007 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:42.510762    4943 scope.go:122] "RemoveContainer" containerID="8866cbc1f1139e188db329cb54c34b31771bd730a2a1ba37f03bd113a853fbcf"
Oct 30 13:19:42.896007 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:42.830579    4943 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-ab3176/simulator-w6-ab3176-kserve-6d9b64549f-dc8xz" podUID="da306792-4f80-4df4-aa44-82991ab856e4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.69:8000/ready\": context deadline exceeded"
Oct 30 13:19:42.896007 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:42.830850    4943 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-d0a18c/distinct-2-w3-d0a18c-kserve-85f8ff4846-rc44r" podUID="d65e47d2-4461-406a-9b71-bb3fac4262f0" containerName="main" probeResult="failure" output="Get \"https://10.132.0.93:8000/ready\": context deadline exceeded"
Oct 30 13:19:42.956955 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:42.956917    4943 patch_prober.go:29] interesting pod/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-fdh7n container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 13:19:42.957333 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:42.957284    4943 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-fdh7n" podUID="8c4e0447-d0df-4536-bea2-00f13e370c5b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:19:43.449312 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:43.449246    4943 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-d0a18c/distinct-w3-d0a18c-kserve-86479f6bbf-5zcqq" podUID="63dace4c-7832-47dd-84ab-2c91483a5563" containerName="main" probeResult="failure" output="Get \"https://10.132.0.89:8000/ready\": context deadline exceeded"
Oct 30 13:19:43.591402 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:43.591346    4943 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w3-d0a18c/distinct-2-w3-d0a18c-kserve-85f8ff4846-rc44r" podUID="d65e47d2-4461-406a-9b71-bb3fac4262f0" containerName="main" probeResult="failure" output="Get \"https://10.132.0.93:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:19:43.725231 ip-10-0-142-84 kubenswrapper[4943]: E1030 13:19:43.725136    4943 kubelet.go:2711] "Housekeeping took longer than expected" err="housekeeping took too long" expected="1s" actual="1.278s"
Oct 30 13:19:43.728287 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:43.725535    4943 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-05942d/distinct-2-w1-05942d-kserve-855c7f7c5f-pwt79"
Oct 30 13:19:43.900436 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:43.900382    4943 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-d0a18c/distinct-2-w3-d0a18c-kserve-85f8ff4846-rc44r" podUID="d65e47d2-4461-406a-9b71-bb3fac4262f0" containerName="main" probeResult="failure" output="Get \"https://10.132.0.93:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:19:43.933286 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:43.912723    4943 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-ab3176/simulator-w6-ab3176-kserve-6d9b64549f-dc8xz" podUID="da306792-4f80-4df4-aa44-82991ab856e4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.69:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:19:43.974290 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:43.964300    4943 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-05942d/distinct-w1-05942d-kserve-5fb44dc664-mcbkc" podUID="d08eb359-cf9e-422c-a3a7-35fd173ab7b6" containerName="main" probeResult="failure" output="Get \"https://10.132.0.91:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:19:43.975099 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:43.975060    4943 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-05942d/distinct-2-w1-05942d-kserve-855c7f7c5f-pwt79" podUID="3dac65dc-7a67-4fdb-840a-765235aaa5bf" containerName="main" probeResult="failure" output="Get \"https://10.132.0.95:8000/ready\": context deadline exceeded"
Oct 30 13:19:44.048253 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:44.048167    4943 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-ab3176/simulator-w6-ab3176-kserve-6d9b64549f-dc8xz"
Oct 30 13:19:44.211285 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:44.203088    4943 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w0-031737-57d4df6b68-xr5s2" podUID="9fc1504d-012b-4afd-bebf-7ac106d6990e" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 13:19:44.271628 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:44.271586    4943 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-fdh7n"]
Oct 30 13:19:44.273875 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:44.273836    4943 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-fdh7n" podUID="8c4e0447-d0df-4536-bea2-00f13e370c5b" containerName="istio-proxy" containerID="cri-o://2bd0afd903ffbbd3b0692efa7b6a8494405b113180956130861bad56145038f8" gracePeriod=30
Oct 30 13:19:44.455700 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:44.455509    4943 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-d0a18c/distinct-w3-d0a18c-kserve-86479f6bbf-5zcqq" podUID="63dace4c-7832-47dd-84ab-2c91483a5563" containerName="main" probeResult="failure" output="Get \"https://10.132.0.89:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:19:44.541059 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:44.539958    4943 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-d0a18c/distinct-2-w3-d0a18c-kserve-85f8ff4846-rc44r"
Oct 30 13:19:44.589559 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:44.589527    4943 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-w2nsg"]
Oct 30 13:19:44.604146 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:44.604105    4943 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-w2nsg" podUID="558c9168-9f89-44f8-845b-74fe5345386c" containerName="istio-proxy" containerID="cri-o://e4527f5cb84ce7b0edaa823d9748c98c31f60d8aca23cc89ef52010097eedfe8" gracePeriod=30
Oct 30 13:19:44.622958 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:44.622882    4943 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-05942d/distinct-2-w1-05942d-kserve-855c7f7c5f-pwt79"
Oct 30 13:19:44.776460 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:44.775749    4943 patch_prober.go:29] interesting pod/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-w2nsg container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 13:19:44.776460 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:44.775845    4943 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-w2nsg" podUID="558c9168-9f89-44f8-845b-74fe5345386c" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 13:19:44.807766 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:44.807727    4943 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-dkztt"]
Oct 30 13:19:44.940225 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:44.937146    4943 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-w5n8g"]
Oct 30 13:19:44.940225 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:44.937380    4943 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-dkztt" podUID="126cf7ee-cae5-4afa-82f5-f36102a35b59" containerName="istio-proxy" containerID="cri-o://678f522b4439205a34af5a0823a35fd4af4579ba13f4fce57df24cfca5331fdc" gracePeriod=30
Oct 30 13:19:44.946360 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:44.945373    4943 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-05942d/distinct-w1-05942d-kserve-5fb44dc664-mcbkc"
Oct 30 13:19:44.946360 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:44.945533    4943 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-w5n8g" podUID="3eba0196-49e5-42ea-8192-ccaac19b1443" containerName="istio-proxy" containerID="cri-o://65a941e112db7eca3ad6e91623540a02f99eceb72c59cea791506645ad7124a1" gracePeriod=30
Oct 30 13:19:44.982920 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:44.982088    4943 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-d0a18c/distinct-w3-d0a18c-kserve-86479f6bbf-5zcqq"
Oct 30 13:19:44.986090 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:44.986059    4943 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-031737/premium-w0-031737-kserve-58d5857cdc-vfnz9"]
Oct 30 13:19:44.987513 ip-10-0-142-84 kubenswrapper[4943]: E1030 13:19:44.987478    4943 cadvisor_stats_provider.go:569] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod79b1a638_b019_4c89_b850_01699ecac9ba.slice/crio-f1bd293d49314f1ab9a0ae336edf53625ec8316216f85a1e76b8d3ebff85780e\": RecentStats: unable to find data in memory cache]"
Oct 30 13:19:44.987684 ip-10-0-142-84 kubenswrapper[4943]: E1030 13:19:44.987656    4943 cadvisor_stats_provider.go:569] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod79b1a638_b019_4c89_b850_01699ecac9ba.slice/crio-f1bd293d49314f1ab9a0ae336edf53625ec8316216f85a1e76b8d3ebff85780e\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod79b1a638_b019_4c89_b850_01699ecac9ba.slice\": RecentStats: unable to find data in memory cache]"
Oct 30 13:19:44.992824 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:44.992800    4943 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-031737/premium-w0-031737-kserve-58d5857cdc-vfnz9"]
Oct 30 13:19:45.079898 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:45.079848    4943 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w1-05942d-7d456fbd84-mvdr9"]
Oct 30 13:19:45.954995 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:45.954964    4943 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-c0194afd-djn2v"]
Oct 30 13:19:45.955640 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:45.955625    4943 cpu_manager.go:414] "containerMap: removing container" podUID="4ffb2c71-a66d-4f4f-8fdd-48707259c348" containerName="revoke-keys"
Oct 30 13:19:45.955680 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:45.955641    4943 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4ffb2c71-a66d-4f4f-8fdd-48707259c348" containerName="revoke-keys"
Oct 30 13:19:45.955712 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:45.955686    4943 cpu_manager.go:414] "containerMap: removing container" podUID="79b1a638-b019-4c89-b850-01699ecac9ba" containerName="main"
Oct 30 13:19:45.955712 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:45.955692    4943 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="79b1a638-b019-4c89-b850-01699ecac9ba" containerName="main"
Oct 30 13:19:45.955712 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:45.955703    4943 cpu_manager.go:414] "containerMap: removing container" podUID="03266586-4be7-42d3-85e5-e607b3754354" containerName="revoke-keys"
Oct 30 13:19:45.955712 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:45.955710    4943 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="03266586-4be7-42d3-85e5-e607b3754354" containerName="revoke-keys"
Oct 30 13:19:45.955832 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:45.955805    4943 memory_manager.go:365] "RemoveStaleState removing state" podUID="79b1a638-b019-4c89-b850-01699ecac9ba" containerName="main"
Oct 30 13:19:45.955832 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:45.955813    4943 memory_manager.go:365] "RemoveStaleState removing state" podUID="03266586-4be7-42d3-85e5-e607b3754354" containerName="revoke-keys"
Oct 30 13:19:45.955832 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:45.955821    4943 memory_manager.go:365] "RemoveStaleState removing state" podUID="4ffb2c71-a66d-4f4f-8fdd-48707259c348" containerName="revoke-keys"
Oct 30 13:19:45.960946 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:45.960924    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c0194afd-djn2v"
Oct 30 13:19:45.970942 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:45.970915    4943 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-67237791-pb7w6"]
Oct 30 13:19:45.975642 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:45.975617    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-67237791-pb7w6"
Oct 30 13:19:45.979918 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:45.979896    4943 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Oct 30 13:19:45.980122 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:45.980107    4943 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Oct 30 13:19:45.980279 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:45.980246    4943 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 13:19:45.981498 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:45.981469    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-c0194afd-djn2v"]
Oct 30 13:19:45.988146 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:45.988127    4943 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-p9z57\""
Oct 30 13:19:45.998339 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:45.998315    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-67237791-pb7w6"]
Oct 30 13:19:46.029527 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:46.029501    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/1b91fb8f-779b-440c-b6a7-d2b89bf020a1-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-c0194afd-djn2v\" (UID: \"1b91fb8f-779b-440c-b6a7-d2b89bf020a1\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c0194afd-djn2v"
Oct 30 13:19:46.029645 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:46.029571    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9crv2\" (UniqueName: \"kubernetes.io/projected/1b91fb8f-779b-440c-b6a7-d2b89bf020a1-kube-api-access-9crv2\") pod \"maas-api-revoke-sub-c0194afd-djn2v\" (UID: \"1b91fb8f-779b-440c-b6a7-d2b89bf020a1\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c0194afd-djn2v"
Oct 30 13:19:46.029746 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:46.029722    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/1b91fb8f-779b-440c-b6a7-d2b89bf020a1-maas-api-xks-ca\") pod \"maas-api-revoke-sub-c0194afd-djn2v\" (UID: \"1b91fb8f-779b-440c-b6a7-d2b89bf020a1\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c0194afd-djn2v"
Oct 30 13:19:46.029873 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:46.029851    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/5a018183-4b6a-4d06-a087-34c2f5168b46-maas-api-xks-ca\") pod \"maas-api-revoke-sub-67237791-pb7w6\" (UID: \"5a018183-4b6a-4d06-a087-34c2f5168b46\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-67237791-pb7w6"
Oct 30 13:19:46.029936 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:46.029911    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9qcn8\" (UniqueName: \"kubernetes.io/projected/5a018183-4b6a-4d06-a087-34c2f5168b46-kube-api-access-9qcn8\") pod \"maas-api-revoke-sub-67237791-pb7w6\" (UID: \"5a018183-4b6a-4d06-a087-34c2f5168b46\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-67237791-pb7w6"
Oct 30 13:19:46.030037 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:46.030023    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/5a018183-4b6a-4d06-a087-34c2f5168b46-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-67237791-pb7w6\" (UID: \"5a018183-4b6a-4d06-a087-34c2f5168b46\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-67237791-pb7w6"
Oct 30 13:19:46.130909 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:46.130878    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/1b91fb8f-779b-440c-b6a7-d2b89bf020a1-maas-api-xks-ca\") pod \"maas-api-revoke-sub-c0194afd-djn2v\" (UID: \"1b91fb8f-779b-440c-b6a7-d2b89bf020a1\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c0194afd-djn2v"
Oct 30 13:19:46.131085 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:46.130948    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/5a018183-4b6a-4d06-a087-34c2f5168b46-maas-api-xks-ca\") pod \"maas-api-revoke-sub-67237791-pb7w6\" (UID: \"5a018183-4b6a-4d06-a087-34c2f5168b46\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-67237791-pb7w6"
Oct 30 13:19:46.131085 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:46.130972    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9qcn8\" (UniqueName: \"kubernetes.io/projected/5a018183-4b6a-4d06-a087-34c2f5168b46-kube-api-access-9qcn8\") pod \"maas-api-revoke-sub-67237791-pb7w6\" (UID: \"5a018183-4b6a-4d06-a087-34c2f5168b46\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-67237791-pb7w6"
Oct 30 13:19:46.131085 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:46.131015    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/5a018183-4b6a-4d06-a087-34c2f5168b46-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-67237791-pb7w6\" (UID: \"5a018183-4b6a-4d06-a087-34c2f5168b46\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-67237791-pb7w6"
Oct 30 13:19:46.131085 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:46.131073    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/1b91fb8f-779b-440c-b6a7-d2b89bf020a1-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-c0194afd-djn2v\" (UID: \"1b91fb8f-779b-440c-b6a7-d2b89bf020a1\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c0194afd-djn2v"
Oct 30 13:19:46.131326 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:46.131130    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9crv2\" (UniqueName: \"kubernetes.io/projected/1b91fb8f-779b-440c-b6a7-d2b89bf020a1-kube-api-access-9crv2\") pod \"maas-api-revoke-sub-c0194afd-djn2v\" (UID: \"1b91fb8f-779b-440c-b6a7-d2b89bf020a1\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c0194afd-djn2v"
Oct 30 13:19:46.131822 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:46.131796    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/1b91fb8f-779b-440c-b6a7-d2b89bf020a1-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-c0194afd-djn2v\" (UID: \"1b91fb8f-779b-440c-b6a7-d2b89bf020a1\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c0194afd-djn2v"
Oct 30 13:19:46.131822 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:46.131809    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/5a018183-4b6a-4d06-a087-34c2f5168b46-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-67237791-pb7w6\" (UID: \"5a018183-4b6a-4d06-a087-34c2f5168b46\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-67237791-pb7w6"
Oct 30 13:19:46.133252 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:46.133230    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/1b91fb8f-779b-440c-b6a7-d2b89bf020a1-maas-api-xks-ca\") pod \"maas-api-revoke-sub-c0194afd-djn2v\" (UID: \"1b91fb8f-779b-440c-b6a7-d2b89bf020a1\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c0194afd-djn2v"
Oct 30 13:19:46.133409 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:46.133391    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/5a018183-4b6a-4d06-a087-34c2f5168b46-maas-api-xks-ca\") pod \"maas-api-revoke-sub-67237791-pb7w6\" (UID: \"5a018183-4b6a-4d06-a087-34c2f5168b46\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-67237791-pb7w6"
Oct 30 13:19:46.156630 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:46.156609    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9crv2\" (UniqueName: \"kubernetes.io/projected/1b91fb8f-779b-440c-b6a7-d2b89bf020a1-kube-api-access-9crv2\") pod \"maas-api-revoke-sub-c0194afd-djn2v\" (UID: \"1b91fb8f-779b-440c-b6a7-d2b89bf020a1\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c0194afd-djn2v"
Oct 30 13:19:46.156753 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:46.156653    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9qcn8\" (UniqueName: \"kubernetes.io/projected/5a018183-4b6a-4d06-a087-34c2f5168b46-kube-api-access-9qcn8\") pod \"maas-api-revoke-sub-67237791-pb7w6\" (UID: \"5a018183-4b6a-4d06-a087-34c2f5168b46\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-67237791-pb7w6"
Oct 30 13:19:46.279794 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:46.279713    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c0194afd-djn2v"
Oct 30 13:19:46.294245 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:46.294218    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-67237791-pb7w6"
Oct 30 13:19:46.358993 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:46.358338    4943 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="79b1a638-b019-4c89-b850-01699ecac9ba" path="/var/lib/kubelet/pods/79b1a638-b019-4c89-b850-01699ecac9ba/volumes"
Oct 30 13:19:46.598999 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:46.598923    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-c0194afd-djn2v"]
Oct 30 13:19:46.610712 ip-10-0-142-84 kubenswrapper[4943]: W1030 13:19:46.610685    4943 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1b91fb8f_779b_440c_b6a7_d2b89bf020a1.slice/crio-d14dcf2604f93d4ebcdf5c265b078700bf2a0fa82cab781b8889788a34306714 WatchSource:0}: Error finding container d14dcf2604f93d4ebcdf5c265b078700bf2a0fa82cab781b8889788a34306714: Status 404 returned error can't find the container with id d14dcf2604f93d4ebcdf5c265b078700bf2a0fa82cab781b8889788a34306714
Oct 30 13:19:46.640846 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:46.640814    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-67237791-pb7w6"]
Oct 30 13:19:47.084452 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:47.084418    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c0194afd-djn2v" event={"ID":"1b91fb8f-779b-440c-b6a7-d2b89bf020a1","Type":"ContainerStarted","Data":"d7bec3d261ca1f991075b9db2facb3d5dd77b621b06514b896c8e78c7f8d19ff"}
Oct 30 13:19:47.084877 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:47.084488    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c0194afd-djn2v" event={"ID":"1b91fb8f-779b-440c-b6a7-d2b89bf020a1","Type":"ContainerStarted","Data":"d14dcf2604f93d4ebcdf5c265b078700bf2a0fa82cab781b8889788a34306714"}
Oct 30 13:19:47.086404 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:47.086380    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-67237791-pb7w6" event={"ID":"5a018183-4b6a-4d06-a087-34c2f5168b46","Type":"ContainerStarted","Data":"d1a4c0870cf8aa8e49e38a17eda5935a5f79583f119a89cd0b5416740b54140c"}
Oct 30 13:19:47.086512 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:47.086426    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-67237791-pb7w6" event={"ID":"5a018183-4b6a-4d06-a087-34c2f5168b46","Type":"ContainerStarted","Data":"bbc54a6850b8dfa18c1a05801dae28872048a98c451bc33c649a4fa401635c97"}
Oct 30 13:19:47.143478 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:47.143389    4943 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c0194afd-djn2v" podStartSLOduration=2.143375478 podStartE2EDuration="2.143375478s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:19:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:19:47.139433967 +0000 UTC m=+1362.927437719" watchObservedRunningTime="2026-10-30 13:19:47.143375478 +0000 UTC m=+1362.931379265"
Oct 30 13:19:47.197471 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:47.197423    4943 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-67237791-pb7w6" podStartSLOduration=2.197409956 podStartE2EDuration="2.197409956s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:19:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:19:47.194852662 +0000 UTC m=+1362.982856394" watchObservedRunningTime="2026-10-30 13:19:47.197409956 +0000 UTC m=+1362.985413685"
Oct 30 13:19:47.450813 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:47.450782    4943 patch_prober.go:29] interesting pod/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-w5n8g container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 13:19:47.450990 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:47.450834    4943 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-w5n8g" podUID="3eba0196-49e5-42ea-8192-ccaac19b1443" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 13:19:47.698111 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:47.698074    4943 patch_prober.go:29] interesting pod/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-dkztt container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 13:19:47.698243 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:47.698132    4943 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-dkztt" podUID="126cf7ee-cae5-4afa-82f5-f36102a35b59" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 13:19:48.095928 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:48.095886    4943 generic.go:356] "Generic (PLEG): container finished" podID="1b91fb8f-779b-440c-b6a7-d2b89bf020a1" containerID="d7bec3d261ca1f991075b9db2facb3d5dd77b621b06514b896c8e78c7f8d19ff" exitCode=0
Oct 30 13:19:48.096371 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:48.095931    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c0194afd-djn2v" event={"ID":"1b91fb8f-779b-440c-b6a7-d2b89bf020a1","Type":"ContainerDied","Data":"d7bec3d261ca1f991075b9db2facb3d5dd77b621b06514b896c8e78c7f8d19ff"}
Oct 30 13:19:48.097579 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:48.097553    4943 generic.go:356] "Generic (PLEG): container finished" podID="5a018183-4b6a-4d06-a087-34c2f5168b46" containerID="d1a4c0870cf8aa8e49e38a17eda5935a5f79583f119a89cd0b5416740b54140c" exitCode=0
Oct 30 13:19:48.097707 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:48.097684    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-67237791-pb7w6" event={"ID":"5a018183-4b6a-4d06-a087-34c2f5168b46","Type":"ContainerDied","Data":"d1a4c0870cf8aa8e49e38a17eda5935a5f79583f119a89cd0b5416740b54140c"}
Oct 30 13:19:49.316309 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:49.316282    4943 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-67237791-pb7w6"
Oct 30 13:19:49.319157 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:49.319136    4943 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c0194afd-djn2v"
Oct 30 13:19:49.392005 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:49.391930    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/1b91fb8f-779b-440c-b6a7-d2b89bf020a1-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/1b91fb8f-779b-440c-b6a7-d2b89bf020a1-maas-api-xks-ca\") pod \"1b91fb8f-779b-440c-b6a7-d2b89bf020a1\" (UID: \"1b91fb8f-779b-440c-b6a7-d2b89bf020a1\") "
Oct 30 13:19:49.392170 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:49.392056    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1b91fb8f-779b-440c-b6a7-d2b89bf020a1-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/1b91fb8f-779b-440c-b6a7-d2b89bf020a1-maas-api-openshift-ca\") pod \"1b91fb8f-779b-440c-b6a7-d2b89bf020a1\" (UID: \"1b91fb8f-779b-440c-b6a7-d2b89bf020a1\") "
Oct 30 13:19:49.392170 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:49.392094    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5a018183-4b6a-4d06-a087-34c2f5168b46-kube-api-access-9qcn8\" (UniqueName: \"kubernetes.io/projected/5a018183-4b6a-4d06-a087-34c2f5168b46-kube-api-access-9qcn8\") pod \"5a018183-4b6a-4d06-a087-34c2f5168b46\" (UID: \"5a018183-4b6a-4d06-a087-34c2f5168b46\") "
Oct 30 13:19:49.392338 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:49.392204    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/5a018183-4b6a-4d06-a087-34c2f5168b46-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/5a018183-4b6a-4d06-a087-34c2f5168b46-maas-api-xks-ca\") pod \"5a018183-4b6a-4d06-a087-34c2f5168b46\" (UID: \"5a018183-4b6a-4d06-a087-34c2f5168b46\") "
Oct 30 13:19:49.392338 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:49.392287    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1b91fb8f-779b-440c-b6a7-d2b89bf020a1-kube-api-access-9crv2\" (UniqueName: \"kubernetes.io/projected/1b91fb8f-779b-440c-b6a7-d2b89bf020a1-kube-api-access-9crv2\") pod \"1b91fb8f-779b-440c-b6a7-d2b89bf020a1\" (UID: \"1b91fb8f-779b-440c-b6a7-d2b89bf020a1\") "
Oct 30 13:19:49.392513 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:49.392379    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5a018183-4b6a-4d06-a087-34c2f5168b46-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/5a018183-4b6a-4d06-a087-34c2f5168b46-maas-api-openshift-ca\") pod \"5a018183-4b6a-4d06-a087-34c2f5168b46\" (UID: \"5a018183-4b6a-4d06-a087-34c2f5168b46\") "
Oct 30 13:19:49.392513 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:49.392480    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1b91fb8f-779b-440c-b6a7-d2b89bf020a1-maas-api-openshift-ca" pod "1b91fb8f-779b-440c-b6a7-d2b89bf020a1" (UID: "1b91fb8f-779b-440c-b6a7-d2b89bf020a1"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 13:19:49.393081 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:49.393055    4943 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/1b91fb8f-779b-440c-b6a7-d2b89bf020a1-maas-api-openshift-ca\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:19:49.394432 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:49.394399    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5a018183-4b6a-4d06-a087-34c2f5168b46-maas-api-openshift-ca" pod "5a018183-4b6a-4d06-a087-34c2f5168b46" (UID: "5a018183-4b6a-4d06-a087-34c2f5168b46"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 13:19:49.395217 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:49.395190    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5a018183-4b6a-4d06-a087-34c2f5168b46-maas-api-xks-ca" pod "5a018183-4b6a-4d06-a087-34c2f5168b46" (UID: "5a018183-4b6a-4d06-a087-34c2f5168b46"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 13:19:49.395354 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:49.395244    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1b91fb8f-779b-440c-b6a7-d2b89bf020a1-maas-api-xks-ca" pod "1b91fb8f-779b-440c-b6a7-d2b89bf020a1" (UID: "1b91fb8f-779b-440c-b6a7-d2b89bf020a1"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 13:19:49.396908 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:49.396879    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5a018183-4b6a-4d06-a087-34c2f5168b46-kube-api-access-9qcn8" pod "5a018183-4b6a-4d06-a087-34c2f5168b46" (UID: "5a018183-4b6a-4d06-a087-34c2f5168b46"). InnerVolumeSpecName "kube-api-access-9qcn8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:19:49.397013 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:49.396936    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1b91fb8f-779b-440c-b6a7-d2b89bf020a1-kube-api-access-9crv2" pod "1b91fb8f-779b-440c-b6a7-d2b89bf020a1" (UID: "1b91fb8f-779b-440c-b6a7-d2b89bf020a1"). InnerVolumeSpecName "kube-api-access-9crv2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:19:49.494877 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:49.494843    4943 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/5a018183-4b6a-4d06-a087-34c2f5168b46-maas-api-openshift-ca\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:19:49.494877 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:49.494872    4943 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/1b91fb8f-779b-440c-b6a7-d2b89bf020a1-maas-api-xks-ca\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:19:49.494877 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:49.494883    4943 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9qcn8\" (UniqueName: \"kubernetes.io/projected/5a018183-4b6a-4d06-a087-34c2f5168b46-kube-api-access-9qcn8\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:19:49.495099 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:49.494891    4943 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/5a018183-4b6a-4d06-a087-34c2f5168b46-maas-api-xks-ca\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:19:49.495099 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:49.494900    4943 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9crv2\" (UniqueName: \"kubernetes.io/projected/1b91fb8f-779b-440c-b6a7-d2b89bf020a1-kube-api-access-9crv2\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:19:49.780844 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:49.780814    4943 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-fdh7n"
Oct 30 13:19:49.799892 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:49.798678    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8c4e0447-d0df-4536-bea2-00f13e370c5b-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8c4e0447-d0df-4536-bea2-00f13e370c5b-istiod-ca-cert\") pod \"8c4e0447-d0df-4536-bea2-00f13e370c5b\" (UID: \"8c4e0447-d0df-4536-bea2-00f13e370c5b\") "
Oct 30 13:19:49.799892 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:49.798724    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8c4e0447-d0df-4536-bea2-00f13e370c5b-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8c4e0447-d0df-4536-bea2-00f13e370c5b-workload-certs\") pod \"8c4e0447-d0df-4536-bea2-00f13e370c5b\" (UID: \"8c4e0447-d0df-4536-bea2-00f13e370c5b\") "
Oct 30 13:19:49.799892 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:49.798756    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8c4e0447-d0df-4536-bea2-00f13e370c5b-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8c4e0447-d0df-4536-bea2-00f13e370c5b-istio-envoy\") pod \"8c4e0447-d0df-4536-bea2-00f13e370c5b\" (UID: \"8c4e0447-d0df-4536-bea2-00f13e370c5b\") "
Oct 30 13:19:49.799892 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:49.798848    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8c4e0447-d0df-4536-bea2-00f13e370c5b-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8c4e0447-d0df-4536-bea2-00f13e370c5b-credential-socket\") pod \"8c4e0447-d0df-4536-bea2-00f13e370c5b\" (UID: \"8c4e0447-d0df-4536-bea2-00f13e370c5b\") "
Oct 30 13:19:49.799892 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:49.798920    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8c4e0447-d0df-4536-bea2-00f13e370c5b-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8c4e0447-d0df-4536-bea2-00f13e370c5b-workload-socket\") pod \"8c4e0447-d0df-4536-bea2-00f13e370c5b\" (UID: \"8c4e0447-d0df-4536-bea2-00f13e370c5b\") "
Oct 30 13:19:49.799892 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:49.799012    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/8c4e0447-d0df-4536-bea2-00f13e370c5b-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8c4e0447-d0df-4536-bea2-00f13e370c5b-istio-podinfo\") pod \"8c4e0447-d0df-4536-bea2-00f13e370c5b\" (UID: \"8c4e0447-d0df-4536-bea2-00f13e370c5b\") "
Oct 30 13:19:49.799892 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:49.799064    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8c4e0447-d0df-4536-bea2-00f13e370c5b-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8c4e0447-d0df-4536-bea2-00f13e370c5b-istio-data\") pod \"8c4e0447-d0df-4536-bea2-00f13e370c5b\" (UID: \"8c4e0447-d0df-4536-bea2-00f13e370c5b\") "
Oct 30 13:19:49.799892 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:49.799165    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8c4e0447-d0df-4536-bea2-00f13e370c5b-istio-token\" (UniqueName: \"kubernetes.io/projected/8c4e0447-d0df-4536-bea2-00f13e370c5b-istio-token\") pod \"8c4e0447-d0df-4536-bea2-00f13e370c5b\" (UID: \"8c4e0447-d0df-4536-bea2-00f13e370c5b\") "
Oct 30 13:19:49.799892 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:49.799203    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8c4e0447-d0df-4536-bea2-00f13e370c5b-kube-api-access-29rrj\" (UniqueName: \"kubernetes.io/projected/8c4e0447-d0df-4536-bea2-00f13e370c5b-kube-api-access-29rrj\") pod \"8c4e0447-d0df-4536-bea2-00f13e370c5b\" (UID: \"8c4e0447-d0df-4536-bea2-00f13e370c5b\") "
Oct 30 13:19:49.799892 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:49.799605    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8c4e0447-d0df-4536-bea2-00f13e370c5b-istio-data" pod "8c4e0447-d0df-4536-bea2-00f13e370c5b" (UID: "8c4e0447-d0df-4536-bea2-00f13e370c5b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:19:49.800549 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:49.800001    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8c4e0447-d0df-4536-bea2-00f13e370c5b-workload-certs" pod "8c4e0447-d0df-4536-bea2-00f13e370c5b" (UID: "8c4e0447-d0df-4536-bea2-00f13e370c5b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:19:49.800549 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:49.800087    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8c4e0447-d0df-4536-bea2-00f13e370c5b-workload-socket" pod "8c4e0447-d0df-4536-bea2-00f13e370c5b" (UID: "8c4e0447-d0df-4536-bea2-00f13e370c5b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:19:49.800549 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:49.800188    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8c4e0447-d0df-4536-bea2-00f13e370c5b-credential-socket" pod "8c4e0447-d0df-4536-bea2-00f13e370c5b" (UID: "8c4e0447-d0df-4536-bea2-00f13e370c5b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:19:49.800549 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:49.800373    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8c4e0447-d0df-4536-bea2-00f13e370c5b-istiod-ca-cert" pod "8c4e0447-d0df-4536-bea2-00f13e370c5b" (UID: "8c4e0447-d0df-4536-bea2-00f13e370c5b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 13:19:49.800549 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:49.800423    4943 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8c4e0447-d0df-4536-bea2-00f13e370c5b-istio-data\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:19:49.801680 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:49.801653    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8c4e0447-d0df-4536-bea2-00f13e370c5b-istio-envoy" pod "8c4e0447-d0df-4536-bea2-00f13e370c5b" (UID: "8c4e0447-d0df-4536-bea2-00f13e370c5b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:19:49.801794 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:49.801777    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8c4e0447-d0df-4536-bea2-00f13e370c5b-istio-token" pod "8c4e0447-d0df-4536-bea2-00f13e370c5b" (UID: "8c4e0447-d0df-4536-bea2-00f13e370c5b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:19:49.802764 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:49.802732    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8c4e0447-d0df-4536-bea2-00f13e370c5b-kube-api-access-29rrj" pod "8c4e0447-d0df-4536-bea2-00f13e370c5b" (UID: "8c4e0447-d0df-4536-bea2-00f13e370c5b"). InnerVolumeSpecName "kube-api-access-29rrj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:19:49.803117 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:49.803093    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/8c4e0447-d0df-4536-bea2-00f13e370c5b-istio-podinfo" pod "8c4e0447-d0df-4536-bea2-00f13e370c5b" (UID: "8c4e0447-d0df-4536-bea2-00f13e370c5b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 13:19:49.920068 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:49.916714    4943 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8c4e0447-d0df-4536-bea2-00f13e370c5b-istio-podinfo\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:19:49.920068 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:49.916748    4943 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8c4e0447-d0df-4536-bea2-00f13e370c5b-istio-token\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:19:49.920068 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:49.916763    4943 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-29rrj\" (UniqueName: \"kubernetes.io/projected/8c4e0447-d0df-4536-bea2-00f13e370c5b-kube-api-access-29rrj\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:19:49.920068 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:49.916981    4943 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8c4e0447-d0df-4536-bea2-00f13e370c5b-istiod-ca-cert\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:19:49.920068 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:49.917005    4943 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8c4e0447-d0df-4536-bea2-00f13e370c5b-workload-certs\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:19:49.920068 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:49.917019    4943 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8c4e0447-d0df-4536-bea2-00f13e370c5b-istio-envoy\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:19:49.920068 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:49.917033    4943 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8c4e0447-d0df-4536-bea2-00f13e370c5b-credential-socket\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:19:49.920068 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:49.917048    4943 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8c4e0447-d0df-4536-bea2-00f13e370c5b-workload-socket\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:19:50.074871 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:50.074843    4943 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-w2nsg"
Oct 30 13:19:50.080645 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:50.080606    4943 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-w1-05942d-7d456fbd84-mvdr9" podUID="7607a99a-237c-4d0c-b5d0-5168ec6a6e3f" containerName="payload-processing" containerID="cri-o://811b2cfece4fbf41e6834879aa348dada91ef3cafecf7876a28a4cf2b00d8d30" gracePeriod=25
Oct 30 13:19:50.120441 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:50.120408    4943 generic.go:356] "Generic (PLEG): container finished" podID="3eba0196-49e5-42ea-8192-ccaac19b1443" containerID="65a941e112db7eca3ad6e91623540a02f99eceb72c59cea791506645ad7124a1" exitCode=0
Oct 30 13:19:50.120580 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:50.120451    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-w5n8g" event={"ID":"3eba0196-49e5-42ea-8192-ccaac19b1443","Type":"ContainerDied","Data":"65a941e112db7eca3ad6e91623540a02f99eceb72c59cea791506645ad7124a1"}
Oct 30 13:19:50.121150 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:50.121123    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/558c9168-9f89-44f8-845b-74fe5345386c-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/558c9168-9f89-44f8-845b-74fe5345386c-workload-certs\") pod \"558c9168-9f89-44f8-845b-74fe5345386c\" (UID: \"558c9168-9f89-44f8-845b-74fe5345386c\") "
Oct 30 13:19:50.121508 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:50.121207    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/558c9168-9f89-44f8-845b-74fe5345386c-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/558c9168-9f89-44f8-845b-74fe5345386c-istio-podinfo\") pod \"558c9168-9f89-44f8-845b-74fe5345386c\" (UID: \"558c9168-9f89-44f8-845b-74fe5345386c\") "
Oct 30 13:19:50.121726 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:50.121700    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/558c9168-9f89-44f8-845b-74fe5345386c-istio-token\" (UniqueName: \"kubernetes.io/projected/558c9168-9f89-44f8-845b-74fe5345386c-istio-token\") pod \"558c9168-9f89-44f8-845b-74fe5345386c\" (UID: \"558c9168-9f89-44f8-845b-74fe5345386c\") "
Oct 30 13:19:50.121817 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:50.121742    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/558c9168-9f89-44f8-845b-74fe5345386c-workload-certs" pod "558c9168-9f89-44f8-845b-74fe5345386c" (UID: "558c9168-9f89-44f8-845b-74fe5345386c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:19:50.121817 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:50.121777    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/558c9168-9f89-44f8-845b-74fe5345386c-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/558c9168-9f89-44f8-845b-74fe5345386c-istio-envoy\") pod \"558c9168-9f89-44f8-845b-74fe5345386c\" (UID: \"558c9168-9f89-44f8-845b-74fe5345386c\") "
Oct 30 13:19:50.121817 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:50.121812    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/558c9168-9f89-44f8-845b-74fe5345386c-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/558c9168-9f89-44f8-845b-74fe5345386c-istiod-ca-cert\") pod \"558c9168-9f89-44f8-845b-74fe5345386c\" (UID: \"558c9168-9f89-44f8-845b-74fe5345386c\") "
Oct 30 13:19:50.121985 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:50.121879    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/558c9168-9f89-44f8-845b-74fe5345386c-kube-api-access-94h9j\" (UniqueName: \"kubernetes.io/projected/558c9168-9f89-44f8-845b-74fe5345386c-kube-api-access-94h9j\") pod \"558c9168-9f89-44f8-845b-74fe5345386c\" (UID: \"558c9168-9f89-44f8-845b-74fe5345386c\") "
Oct 30 13:19:50.121985 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:50.121931    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/558c9168-9f89-44f8-845b-74fe5345386c-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/558c9168-9f89-44f8-845b-74fe5345386c-credential-socket\") pod \"558c9168-9f89-44f8-845b-74fe5345386c\" (UID: \"558c9168-9f89-44f8-845b-74fe5345386c\") "
Oct 30 13:19:50.122102 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:50.122043    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/558c9168-9f89-44f8-845b-74fe5345386c-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/558c9168-9f89-44f8-845b-74fe5345386c-workload-socket\") pod \"558c9168-9f89-44f8-845b-74fe5345386c\" (UID: \"558c9168-9f89-44f8-845b-74fe5345386c\") "
Oct 30 13:19:50.122102 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:50.122091    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/558c9168-9f89-44f8-845b-74fe5345386c-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/558c9168-9f89-44f8-845b-74fe5345386c-istio-data\") pod \"558c9168-9f89-44f8-845b-74fe5345386c\" (UID: \"558c9168-9f89-44f8-845b-74fe5345386c\") "
Oct 30 13:19:50.122532 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:50.122513    4943 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/558c9168-9f89-44f8-845b-74fe5345386c-workload-certs\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:19:50.123047 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:50.122970    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/558c9168-9f89-44f8-845b-74fe5345386c-istio-data" pod "558c9168-9f89-44f8-845b-74fe5345386c" (UID: "558c9168-9f89-44f8-845b-74fe5345386c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:19:50.123047 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:50.122992    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/558c9168-9f89-44f8-845b-74fe5345386c-credential-socket" pod "558c9168-9f89-44f8-845b-74fe5345386c" (UID: "558c9168-9f89-44f8-845b-74fe5345386c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:19:50.123302 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:50.123233    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c0194afd-djn2v" event={"ID":"1b91fb8f-779b-440c-b6a7-d2b89bf020a1","Type":"ContainerDied","Data":"d14dcf2604f93d4ebcdf5c265b078700bf2a0fa82cab781b8889788a34306714"}
Oct 30 13:19:50.123398 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:50.123304    4943 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d14dcf2604f93d4ebcdf5c265b078700bf2a0fa82cab781b8889788a34306714"
Oct 30 13:19:50.123398 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:50.123354    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/558c9168-9f89-44f8-845b-74fe5345386c-istiod-ca-cert" pod "558c9168-9f89-44f8-845b-74fe5345386c" (UID: "558c9168-9f89-44f8-845b-74fe5345386c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 13:19:50.123398 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:50.123386    4943 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c0194afd-djn2v"
Oct 30 13:19:50.123567 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:50.123422    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/558c9168-9f89-44f8-845b-74fe5345386c-workload-socket" pod "558c9168-9f89-44f8-845b-74fe5345386c" (UID: "558c9168-9f89-44f8-845b-74fe5345386c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:19:50.129285 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:50.129181    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/558c9168-9f89-44f8-845b-74fe5345386c-istio-envoy" pod "558c9168-9f89-44f8-845b-74fe5345386c" (UID: "558c9168-9f89-44f8-845b-74fe5345386c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:19:50.129285 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:50.129190    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/558c9168-9f89-44f8-845b-74fe5345386c-istio-podinfo" pod "558c9168-9f89-44f8-845b-74fe5345386c" (UID: "558c9168-9f89-44f8-845b-74fe5345386c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 13:19:50.129285 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:50.129233    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/558c9168-9f89-44f8-845b-74fe5345386c-kube-api-access-94h9j" pod "558c9168-9f89-44f8-845b-74fe5345386c" (UID: "558c9168-9f89-44f8-845b-74fe5345386c"). InnerVolumeSpecName "kube-api-access-94h9j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:19:50.129593 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:50.129401    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/558c9168-9f89-44f8-845b-74fe5345386c-istio-token" pod "558c9168-9f89-44f8-845b-74fe5345386c" (UID: "558c9168-9f89-44f8-845b-74fe5345386c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:19:50.130502 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:50.130478    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-67237791-pb7w6" event={"ID":"5a018183-4b6a-4d06-a087-34c2f5168b46","Type":"ContainerDied","Data":"bbc54a6850b8dfa18c1a05801dae28872048a98c451bc33c649a4fa401635c97"}
Oct 30 13:19:50.130643 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:50.130630    4943 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="bbc54a6850b8dfa18c1a05801dae28872048a98c451bc33c649a4fa401635c97"
Oct 30 13:19:50.130738 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:50.130727    4943 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-67237791-pb7w6"
Oct 30 13:19:50.140450 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:50.140425    4943 generic.go:356] "Generic (PLEG): container finished" podID="126cf7ee-cae5-4afa-82f5-f36102a35b59" containerID="678f522b4439205a34af5a0823a35fd4af4579ba13f4fce57df24cfca5331fdc" exitCode=0
Oct 30 13:19:50.140552 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:50.140489    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-dkztt" event={"ID":"126cf7ee-cae5-4afa-82f5-f36102a35b59","Type":"ContainerDied","Data":"678f522b4439205a34af5a0823a35fd4af4579ba13f4fce57df24cfca5331fdc"}
Oct 30 13:19:50.147396 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:50.147368    4943 generic.go:356] "Generic (PLEG): container finished" podID="8c4e0447-d0df-4536-bea2-00f13e370c5b" containerID="2bd0afd903ffbbd3b0692efa7b6a8494405b113180956130861bad56145038f8" exitCode=0
Oct 30 13:19:50.147510 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:50.147486    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-fdh7n" event={"ID":"8c4e0447-d0df-4536-bea2-00f13e370c5b","Type":"ContainerDied","Data":"2bd0afd903ffbbd3b0692efa7b6a8494405b113180956130861bad56145038f8"}
Oct 30 13:19:50.147575 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:50.147525    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-fdh7n" event={"ID":"8c4e0447-d0df-4536-bea2-00f13e370c5b","Type":"ContainerDied","Data":"89d4728d41eda972cca81436a97795f4e831de63c9981162f9051d6ef8533076"}
Oct 30 13:19:50.147575 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:50.147552    4943 scope.go:122] "RemoveContainer" containerID="2bd0afd903ffbbd3b0692efa7b6a8494405b113180956130861bad56145038f8"
Oct 30 13:19:50.147750 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:50.147730    4943 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-fdh7n"
Oct 30 13:19:50.152323 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:50.152294    4943 generic.go:356] "Generic (PLEG): container finished" podID="558c9168-9f89-44f8-845b-74fe5345386c" containerID="e4527f5cb84ce7b0edaa823d9748c98c31f60d8aca23cc89ef52010097eedfe8" exitCode=0
Oct 30 13:19:50.152425 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:50.152362    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-w2nsg" event={"ID":"558c9168-9f89-44f8-845b-74fe5345386c","Type":"ContainerDied","Data":"e4527f5cb84ce7b0edaa823d9748c98c31f60d8aca23cc89ef52010097eedfe8"}
Oct 30 13:19:50.152425 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:50.152402    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-w2nsg" event={"ID":"558c9168-9f89-44f8-845b-74fe5345386c","Type":"ContainerDied","Data":"18b4fee8f9edc8f1be6601214884ed6202b2a186ecef5b89dfabdf3f6a92bfff"}
Oct 30 13:19:50.152628 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:50.152474    4943 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-w2nsg"
Oct 30 13:19:50.174431 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:50.174411    4943 scope.go:122] "RemoveContainer" containerID="f8327c30b96218ee8c871b6cded44493a4bbe2ef4d8aeacae77de64292f0f756"
Oct 30 13:19:50.176224 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:50.176200    4943 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-dkztt"
Oct 30 13:19:50.189698 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:50.189643    4943 scope.go:122] "RemoveContainer" containerID="2bd0afd903ffbbd3b0692efa7b6a8494405b113180956130861bad56145038f8"
Oct 30 13:19:50.189966 ip-10-0-142-84 kubenswrapper[4943]: E1030 13:19:50.189945    4943 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2bd0afd903ffbbd3b0692efa7b6a8494405b113180956130861bad56145038f8\": container with ID starting with 2bd0afd903ffbbd3b0692efa7b6a8494405b113180956130861bad56145038f8 not found: ID does not exist" containerID="2bd0afd903ffbbd3b0692efa7b6a8494405b113180956130861bad56145038f8"
Oct 30 13:19:50.190045 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:50.189982    4943 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2bd0afd903ffbbd3b0692efa7b6a8494405b113180956130861bad56145038f8"} err="failed to get container status \"2bd0afd903ffbbd3b0692efa7b6a8494405b113180956130861bad56145038f8\": rpc error: code = NotFound desc = could not find container \"2bd0afd903ffbbd3b0692efa7b6a8494405b113180956130861bad56145038f8\": container with ID starting with 2bd0afd903ffbbd3b0692efa7b6a8494405b113180956130861bad56145038f8 not found: ID does not exist"
Oct 30 13:19:50.190045 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:50.190000    4943 scope.go:122] "RemoveContainer" containerID="f8327c30b96218ee8c871b6cded44493a4bbe2ef4d8aeacae77de64292f0f756"
Oct 30 13:19:50.190297 ip-10-0-142-84 kubenswrapper[4943]: E1030 13:19:50.190249    4943 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f8327c30b96218ee8c871b6cded44493a4bbe2ef4d8aeacae77de64292f0f756\": container with ID starting with f8327c30b96218ee8c871b6cded44493a4bbe2ef4d8aeacae77de64292f0f756 not found: ID does not exist" containerID="f8327c30b96218ee8c871b6cded44493a4bbe2ef4d8aeacae77de64292f0f756"
Oct 30 13:19:50.190397 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:50.190304    4943 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f8327c30b96218ee8c871b6cded44493a4bbe2ef4d8aeacae77de64292f0f756"} err="failed to get container status \"f8327c30b96218ee8c871b6cded44493a4bbe2ef4d8aeacae77de64292f0f756\": rpc error: code = NotFound desc = could not find container \"f8327c30b96218ee8c871b6cded44493a4bbe2ef4d8aeacae77de64292f0f756\": container with ID starting with f8327c30b96218ee8c871b6cded44493a4bbe2ef4d8aeacae77de64292f0f756 not found: ID does not exist"
Oct 30 13:19:50.190397 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:50.190319    4943 scope.go:122] "RemoveContainer" containerID="e4527f5cb84ce7b0edaa823d9748c98c31f60d8aca23cc89ef52010097eedfe8"
Oct 30 13:19:50.200794 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:50.200769    4943 scope.go:122] "RemoveContainer" containerID="e4527f5cb84ce7b0edaa823d9748c98c31f60d8aca23cc89ef52010097eedfe8"
Oct 30 13:19:50.201067 ip-10-0-142-84 kubenswrapper[4943]: E1030 13:19:50.201048    4943 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e4527f5cb84ce7b0edaa823d9748c98c31f60d8aca23cc89ef52010097eedfe8\": container with ID starting with e4527f5cb84ce7b0edaa823d9748c98c31f60d8aca23cc89ef52010097eedfe8 not found: ID does not exist" containerID="e4527f5cb84ce7b0edaa823d9748c98c31f60d8aca23cc89ef52010097eedfe8"
Oct 30 13:19:50.201140 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:50.201072    4943 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e4527f5cb84ce7b0edaa823d9748c98c31f60d8aca23cc89ef52010097eedfe8"} err="failed to get container status \"e4527f5cb84ce7b0edaa823d9748c98c31f60d8aca23cc89ef52010097eedfe8\": rpc error: code = NotFound desc = could not find container \"e4527f5cb84ce7b0edaa823d9748c98c31f60d8aca23cc89ef52010097eedfe8\": container with ID starting with e4527f5cb84ce7b0edaa823d9748c98c31f60d8aca23cc89ef52010097eedfe8 not found: ID does not exist"
Oct 30 13:19:50.216432 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:50.216414    4943 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-w5n8g"
Oct 30 13:19:50.223256 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:50.223232    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/126cf7ee-cae5-4afa-82f5-f36102a35b59-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/126cf7ee-cae5-4afa-82f5-f36102a35b59-istiod-ca-cert\") pod \"126cf7ee-cae5-4afa-82f5-f36102a35b59\" (UID: \"126cf7ee-cae5-4afa-82f5-f36102a35b59\") "
Oct 30 13:19:50.223356 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:50.223326    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/126cf7ee-cae5-4afa-82f5-f36102a35b59-istio-token\" (UniqueName: \"kubernetes.io/projected/126cf7ee-cae5-4afa-82f5-f36102a35b59-istio-token\") pod \"126cf7ee-cae5-4afa-82f5-f36102a35b59\" (UID: \"126cf7ee-cae5-4afa-82f5-f36102a35b59\") "
Oct 30 13:19:50.223405 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:50.223382    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/126cf7ee-cae5-4afa-82f5-f36102a35b59-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/126cf7ee-cae5-4afa-82f5-f36102a35b59-workload-certs\") pod \"126cf7ee-cae5-4afa-82f5-f36102a35b59\" (UID: \"126cf7ee-cae5-4afa-82f5-f36102a35b59\") "
Oct 30 13:19:50.223471 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:50.223456    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/126cf7ee-cae5-4afa-82f5-f36102a35b59-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/126cf7ee-cae5-4afa-82f5-f36102a35b59-istio-envoy\") pod \"126cf7ee-cae5-4afa-82f5-f36102a35b59\" (UID: \"126cf7ee-cae5-4afa-82f5-f36102a35b59\") "
Oct 30 13:19:50.223509 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:50.223499    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/126cf7ee-cae5-4afa-82f5-f36102a35b59-kube-api-access-gj4s9\" (UniqueName: \"kubernetes.io/projected/126cf7ee-cae5-4afa-82f5-f36102a35b59-kube-api-access-gj4s9\") pod \"126cf7ee-cae5-4afa-82f5-f36102a35b59\" (UID: \"126cf7ee-cae5-4afa-82f5-f36102a35b59\") "
Oct 30 13:19:50.223540 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:50.223531    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/126cf7ee-cae5-4afa-82f5-f36102a35b59-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/126cf7ee-cae5-4afa-82f5-f36102a35b59-istio-podinfo\") pod \"126cf7ee-cae5-4afa-82f5-f36102a35b59\" (UID: \"126cf7ee-cae5-4afa-82f5-f36102a35b59\") "
Oct 30 13:19:50.223612 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:50.223585    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/126cf7ee-cae5-4afa-82f5-f36102a35b59-istiod-ca-cert" pod "126cf7ee-cae5-4afa-82f5-f36102a35b59" (UID: "126cf7ee-cae5-4afa-82f5-f36102a35b59"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 13:19:50.223745 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:50.223718    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/126cf7ee-cae5-4afa-82f5-f36102a35b59-workload-certs" pod "126cf7ee-cae5-4afa-82f5-f36102a35b59" (UID: "126cf7ee-cae5-4afa-82f5-f36102a35b59"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:19:50.223812 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:50.223755    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/126cf7ee-cae5-4afa-82f5-f36102a35b59-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/126cf7ee-cae5-4afa-82f5-f36102a35b59-workload-socket\") pod \"126cf7ee-cae5-4afa-82f5-f36102a35b59\" (UID: \"126cf7ee-cae5-4afa-82f5-f36102a35b59\") "
Oct 30 13:19:50.223867 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:50.223830    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/126cf7ee-cae5-4afa-82f5-f36102a35b59-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/126cf7ee-cae5-4afa-82f5-f36102a35b59-istio-data\") pod \"126cf7ee-cae5-4afa-82f5-f36102a35b59\" (UID: \"126cf7ee-cae5-4afa-82f5-f36102a35b59\") "
Oct 30 13:19:50.223930 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:50.223899    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/126cf7ee-cae5-4afa-82f5-f36102a35b59-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/126cf7ee-cae5-4afa-82f5-f36102a35b59-credential-socket\") pod \"126cf7ee-cae5-4afa-82f5-f36102a35b59\" (UID: \"126cf7ee-cae5-4afa-82f5-f36102a35b59\") "
Oct 30 13:19:50.224043 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:50.224020    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/126cf7ee-cae5-4afa-82f5-f36102a35b59-workload-socket" pod "126cf7ee-cae5-4afa-82f5-f36102a35b59" (UID: "126cf7ee-cae5-4afa-82f5-f36102a35b59"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:19:50.224172 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:50.224150    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/126cf7ee-cae5-4afa-82f5-f36102a35b59-istio-data" pod "126cf7ee-cae5-4afa-82f5-f36102a35b59" (UID: "126cf7ee-cae5-4afa-82f5-f36102a35b59"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:19:50.224317 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:50.224296    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/126cf7ee-cae5-4afa-82f5-f36102a35b59-credential-socket" pod "126cf7ee-cae5-4afa-82f5-f36102a35b59" (UID: "126cf7ee-cae5-4afa-82f5-f36102a35b59"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:19:50.224681 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:50.224660    4943 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/558c9168-9f89-44f8-845b-74fe5345386c-credential-socket\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:19:50.224766 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:50.224686    4943 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/126cf7ee-cae5-4afa-82f5-f36102a35b59-credential-socket\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:19:50.224766 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:50.224702    4943 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/558c9168-9f89-44f8-845b-74fe5345386c-workload-socket\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:19:50.224766 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:50.224732    4943 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/558c9168-9f89-44f8-845b-74fe5345386c-istio-data\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:19:50.224766 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:50.224749    4943 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/558c9168-9f89-44f8-845b-74fe5345386c-istio-podinfo\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:19:50.224766 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:50.224763    4943 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/126cf7ee-cae5-4afa-82f5-f36102a35b59-istiod-ca-cert\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:19:50.225052 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:50.224776    4943 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/126cf7ee-cae5-4afa-82f5-f36102a35b59-workload-certs\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:19:50.225052 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:50.224789    4943 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/558c9168-9f89-44f8-845b-74fe5345386c-istio-token\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:19:50.225052 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:50.224802    4943 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/558c9168-9f89-44f8-845b-74fe5345386c-istio-envoy\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:19:50.225052 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:50.224815    4943 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/558c9168-9f89-44f8-845b-74fe5345386c-istiod-ca-cert\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:19:50.225052 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:50.224829    4943 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/126cf7ee-cae5-4afa-82f5-f36102a35b59-workload-socket\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:19:50.225052 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:50.224842    4943 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-94h9j\" (UniqueName: \"kubernetes.io/projected/558c9168-9f89-44f8-845b-74fe5345386c-kube-api-access-94h9j\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:19:50.225052 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:50.224864    4943 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/126cf7ee-cae5-4afa-82f5-f36102a35b59-istio-data\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:19:50.225920 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:50.225891    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/126cf7ee-cae5-4afa-82f5-f36102a35b59-istio-envoy" pod "126cf7ee-cae5-4afa-82f5-f36102a35b59" (UID: "126cf7ee-cae5-4afa-82f5-f36102a35b59"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:19:50.226014 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:50.225916    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/126cf7ee-cae5-4afa-82f5-f36102a35b59-istio-token" pod "126cf7ee-cae5-4afa-82f5-f36102a35b59" (UID: "126cf7ee-cae5-4afa-82f5-f36102a35b59"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:19:50.226014 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:50.225950    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/126cf7ee-cae5-4afa-82f5-f36102a35b59-istio-podinfo" pod "126cf7ee-cae5-4afa-82f5-f36102a35b59" (UID: "126cf7ee-cae5-4afa-82f5-f36102a35b59"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 13:19:50.226152 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:50.226138    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/126cf7ee-cae5-4afa-82f5-f36102a35b59-kube-api-access-gj4s9" pod "126cf7ee-cae5-4afa-82f5-f36102a35b59" (UID: "126cf7ee-cae5-4afa-82f5-f36102a35b59"). InnerVolumeSpecName "kube-api-access-gj4s9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:19:50.272972 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:50.272947    4943 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-w2nsg"]
Oct 30 13:19:50.280026 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:50.279998    4943 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-w2nsg"]
Oct 30 13:19:50.325712 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:50.325682    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3eba0196-49e5-42ea-8192-ccaac19b1443-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3eba0196-49e5-42ea-8192-ccaac19b1443-credential-socket\") pod \"3eba0196-49e5-42ea-8192-ccaac19b1443\" (UID: \"3eba0196-49e5-42ea-8192-ccaac19b1443\") "
Oct 30 13:19:50.326091 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:50.325743    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3eba0196-49e5-42ea-8192-ccaac19b1443-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3eba0196-49e5-42ea-8192-ccaac19b1443-istio-data\") pod \"3eba0196-49e5-42ea-8192-ccaac19b1443\" (UID: \"3eba0196-49e5-42ea-8192-ccaac19b1443\") "
Oct 30 13:19:50.326091 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:50.325776    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3eba0196-49e5-42ea-8192-ccaac19b1443-istio-token\" (UniqueName: \"kubernetes.io/projected/3eba0196-49e5-42ea-8192-ccaac19b1443-istio-token\") pod \"3eba0196-49e5-42ea-8192-ccaac19b1443\" (UID: \"3eba0196-49e5-42ea-8192-ccaac19b1443\") "
Oct 30 13:19:50.326091 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:50.325805    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3eba0196-49e5-42ea-8192-ccaac19b1443-kube-api-access-bkg9j\" (UniqueName: \"kubernetes.io/projected/3eba0196-49e5-42ea-8192-ccaac19b1443-kube-api-access-bkg9j\") pod \"3eba0196-49e5-42ea-8192-ccaac19b1443\" (UID: \"3eba0196-49e5-42ea-8192-ccaac19b1443\") "
Oct 30 13:19:50.326091 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:50.325856    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3eba0196-49e5-42ea-8192-ccaac19b1443-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3eba0196-49e5-42ea-8192-ccaac19b1443-workload-socket\") pod \"3eba0196-49e5-42ea-8192-ccaac19b1443\" (UID: \"3eba0196-49e5-42ea-8192-ccaac19b1443\") "
Oct 30 13:19:50.326091 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:50.325920    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3eba0196-49e5-42ea-8192-ccaac19b1443-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3eba0196-49e5-42ea-8192-ccaac19b1443-istiod-ca-cert\") pod \"3eba0196-49e5-42ea-8192-ccaac19b1443\" (UID: \"3eba0196-49e5-42ea-8192-ccaac19b1443\") "
Oct 30 13:19:50.326091 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:50.325951    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3eba0196-49e5-42ea-8192-ccaac19b1443-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3eba0196-49e5-42ea-8192-ccaac19b1443-istio-envoy\") pod \"3eba0196-49e5-42ea-8192-ccaac19b1443\" (UID: \"3eba0196-49e5-42ea-8192-ccaac19b1443\") "
Oct 30 13:19:50.326091 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:50.325950    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3eba0196-49e5-42ea-8192-ccaac19b1443-credential-socket" pod "3eba0196-49e5-42ea-8192-ccaac19b1443" (UID: "3eba0196-49e5-42ea-8192-ccaac19b1443"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:19:50.326091 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:50.325978    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3eba0196-49e5-42ea-8192-ccaac19b1443-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3eba0196-49e5-42ea-8192-ccaac19b1443-workload-certs\") pod \"3eba0196-49e5-42ea-8192-ccaac19b1443\" (UID: \"3eba0196-49e5-42ea-8192-ccaac19b1443\") "
Oct 30 13:19:50.326091 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:50.325986    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3eba0196-49e5-42ea-8192-ccaac19b1443-istio-data" pod "3eba0196-49e5-42ea-8192-ccaac19b1443" (UID: "3eba0196-49e5-42ea-8192-ccaac19b1443"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:19:50.326091 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:50.326046    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/3eba0196-49e5-42ea-8192-ccaac19b1443-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3eba0196-49e5-42ea-8192-ccaac19b1443-istio-podinfo\") pod \"3eba0196-49e5-42ea-8192-ccaac19b1443\" (UID: \"3eba0196-49e5-42ea-8192-ccaac19b1443\") "
Oct 30 13:19:50.326642 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:50.326229    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3eba0196-49e5-42ea-8192-ccaac19b1443-workload-certs" pod "3eba0196-49e5-42ea-8192-ccaac19b1443" (UID: "3eba0196-49e5-42ea-8192-ccaac19b1443"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:19:50.326642 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:50.326360    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3eba0196-49e5-42ea-8192-ccaac19b1443-istiod-ca-cert" pod "3eba0196-49e5-42ea-8192-ccaac19b1443" (UID: "3eba0196-49e5-42ea-8192-ccaac19b1443"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 13:19:50.326642 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:50.326479    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3eba0196-49e5-42ea-8192-ccaac19b1443-workload-socket" pod "3eba0196-49e5-42ea-8192-ccaac19b1443" (UID: "3eba0196-49e5-42ea-8192-ccaac19b1443"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:19:50.326793 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:50.326685    4943 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/126cf7ee-cae5-4afa-82f5-f36102a35b59-istio-envoy\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:19:50.326793 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:50.326705    4943 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gj4s9\" (UniqueName: \"kubernetes.io/projected/126cf7ee-cae5-4afa-82f5-f36102a35b59-kube-api-access-gj4s9\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:19:50.326793 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:50.326719    4943 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/126cf7ee-cae5-4afa-82f5-f36102a35b59-istio-podinfo\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:19:50.326793 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:50.326733    4943 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3eba0196-49e5-42ea-8192-ccaac19b1443-credential-socket\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:19:50.326793 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:50.326746    4943 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3eba0196-49e5-42ea-8192-ccaac19b1443-istio-data\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:19:50.326793 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:50.326758    4943 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3eba0196-49e5-42ea-8192-ccaac19b1443-workload-socket\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:19:50.326793 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:50.326771    4943 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3eba0196-49e5-42ea-8192-ccaac19b1443-istiod-ca-cert\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:19:50.326793 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:50.326786    4943 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3eba0196-49e5-42ea-8192-ccaac19b1443-workload-certs\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:19:50.327202 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:50.326798    4943 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/126cf7ee-cae5-4afa-82f5-f36102a35b59-istio-token\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:19:50.328237 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:50.328204    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3eba0196-49e5-42ea-8192-ccaac19b1443-istio-token" pod "3eba0196-49e5-42ea-8192-ccaac19b1443" (UID: "3eba0196-49e5-42ea-8192-ccaac19b1443"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:19:50.328237 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:50.328217    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3eba0196-49e5-42ea-8192-ccaac19b1443-kube-api-access-bkg9j" pod "3eba0196-49e5-42ea-8192-ccaac19b1443" (UID: "3eba0196-49e5-42ea-8192-ccaac19b1443"). InnerVolumeSpecName "kube-api-access-bkg9j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:19:50.328555 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:50.328534    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3eba0196-49e5-42ea-8192-ccaac19b1443-istio-envoy" pod "3eba0196-49e5-42ea-8192-ccaac19b1443" (UID: "3eba0196-49e5-42ea-8192-ccaac19b1443"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:19:50.328626 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:50.328573    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/3eba0196-49e5-42ea-8192-ccaac19b1443-istio-podinfo" pod "3eba0196-49e5-42ea-8192-ccaac19b1443" (UID: "3eba0196-49e5-42ea-8192-ccaac19b1443"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 13:19:50.329412 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:50.329390    4943 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-fdh7n"]
Oct 30 13:19:50.336303 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:50.336279    4943 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-fdh7n"]
Oct 30 13:19:50.351564 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:50.351544    4943 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="558c9168-9f89-44f8-845b-74fe5345386c" path="/var/lib/kubelet/pods/558c9168-9f89-44f8-845b-74fe5345386c/volumes"
Oct 30 13:19:50.352029 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:50.352016    4943 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8c4e0447-d0df-4536-bea2-00f13e370c5b" path="/var/lib/kubelet/pods/8c4e0447-d0df-4536-bea2-00f13e370c5b/volumes"
Oct 30 13:19:50.428151 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:50.428118    4943 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3eba0196-49e5-42ea-8192-ccaac19b1443-istio-token\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:19:50.428386 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:50.428154    4943 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bkg9j\" (UniqueName: \"kubernetes.io/projected/3eba0196-49e5-42ea-8192-ccaac19b1443-kube-api-access-bkg9j\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:19:50.428386 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:50.428171    4943 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3eba0196-49e5-42ea-8192-ccaac19b1443-istio-envoy\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:19:50.428386 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:50.428185    4943 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3eba0196-49e5-42ea-8192-ccaac19b1443-istio-podinfo\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:19:51.028505 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:51.028469    4943 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w1-05942d-7d456fbd84-mvdr9" podUID="7607a99a-237c-4d0c-b5d0-5168ec6a6e3f" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 13:19:51.163028 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:51.162998    4943 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-w5n8g"
Oct 30 13:19:51.163198 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:51.163040    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-w5n8g" event={"ID":"3eba0196-49e5-42ea-8192-ccaac19b1443","Type":"ContainerDied","Data":"70072baea518cb98cc23ae5ae41eaa2c8764ad46629f268cdc454cc801c8163a"}
Oct 30 13:19:51.163198 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:51.163111    4943 scope.go:122] "RemoveContainer" containerID="65a941e112db7eca3ad6e91623540a02f99eceb72c59cea791506645ad7124a1"
Oct 30 13:19:51.165071 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:51.165043    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-dkztt" event={"ID":"126cf7ee-cae5-4afa-82f5-f36102a35b59","Type":"ContainerDied","Data":"f5cc62c06cbf340682f1a435ebbd8f4dfd089d98d1e0b3316a15b246d6bee208"}
Oct 30 13:19:51.165238 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:51.165097    4943 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-dkztt"
Oct 30 13:19:51.176026 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:51.176002    4943 scope.go:122] "RemoveContainer" containerID="678f522b4439205a34af5a0823a35fd4af4579ba13f4fce57df24cfca5331fdc"
Oct 30 13:19:51.224575 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:51.224552    4943 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-w5n8g"]
Oct 30 13:19:51.237884 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:51.237859    4943 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-w5n8g"]
Oct 30 13:19:51.270818 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:51.270769    4943 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-dkztt"]
Oct 30 13:19:51.275710 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:51.275685    4943 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-dkztt"]
Oct 30 13:19:52.351280 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:52.351238    4943 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="126cf7ee-cae5-4afa-82f5-f36102a35b59" path="/var/lib/kubelet/pods/126cf7ee-cae5-4afa-82f5-f36102a35b59/volumes"
Oct 30 13:19:52.351763 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:52.351746    4943 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="3eba0196-49e5-42ea-8192-ccaac19b1443" path="/var/lib/kubelet/pods/3eba0196-49e5-42ea-8192-ccaac19b1443/volumes"
Oct 30 13:19:53.944770 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:53.944702    4943 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w1-05942d/distinct-w1-05942d-kserve-5fb44dc664-mcbkc" podUID="d08eb359-cf9e-422c-a3a7-35fd173ab7b6" containerName="main" containerID="cri-o://5c0097f27d893027b33c144a75796ecbad447f17006917035307e622e520f2d7" gracePeriod=45
Oct 30 13:19:53.968541 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:53.968510    4943 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w0-031737-57d4df6b68-xr5s2" podUID="9fc1504d-012b-4afd-bebf-7ac106d6990e" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 13:19:54.017552 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:54.017501    4943 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w1-05942d/distinct-2-w1-05942d-kserve-855c7f7c5f-pwt79" podUID="3dac65dc-7a67-4fdb-840a-765235aaa5bf" containerName="main" containerID="cri-o://632ff98e6b4cf4c86974d5cbbb1124f5f9e7eeacca5c2759c012e3e910264255" gracePeriod=45
Oct 30 13:19:54.133516 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:54.133482    4943 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-05942d/distinct-w1-05942d-kserve-5fb44dc664-mcbkc" podUID="d08eb359-cf9e-422c-a3a7-35fd173ab7b6" containerName="main" probeResult="failure" output="Get \"https://10.132.0.91:8000/ready\": dial tcp 10.132.0.91:8000: connect: connection refused"
Oct 30 13:19:54.211354 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:54.206956    4943 generic.go:356] "Generic (PLEG): container finished" podID="3dac65dc-7a67-4fdb-840a-765235aaa5bf" containerID="632ff98e6b4cf4c86974d5cbbb1124f5f9e7eeacca5c2759c012e3e910264255" exitCode=0
Oct 30 13:19:54.211354 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:54.207172    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-05942d/distinct-2-w1-05942d-kserve-855c7f7c5f-pwt79" event={"ID":"3dac65dc-7a67-4fdb-840a-765235aaa5bf","Type":"ContainerDied","Data":"632ff98e6b4cf4c86974d5cbbb1124f5f9e7eeacca5c2759c012e3e910264255"}
Oct 30 13:19:54.211354 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:54.209381    4943 generic.go:356] "Generic (PLEG): container finished" podID="d08eb359-cf9e-422c-a3a7-35fd173ab7b6" containerID="5c0097f27d893027b33c144a75796ecbad447f17006917035307e622e520f2d7" exitCode=0
Oct 30 13:19:54.211354 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:54.209579    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-05942d/distinct-w1-05942d-kserve-5fb44dc664-mcbkc" event={"ID":"d08eb359-cf9e-422c-a3a7-35fd173ab7b6","Type":"ContainerDied","Data":"5c0097f27d893027b33c144a75796ecbad447f17006917035307e622e520f2d7"}
Oct 30 13:19:54.211887 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:54.211690    4943 generic.go:356] "Generic (PLEG): container finished" podID="9fc1504d-012b-4afd-bebf-7ac106d6990e" containerID="2f655124e975981258127d10703872d21c94f5f1dedd7ae7c29ca637f5fbbd4b" exitCode=0
Oct 30 13:19:54.211887 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:54.211753    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w0-031737-57d4df6b68-xr5s2" event={"ID":"9fc1504d-012b-4afd-bebf-7ac106d6990e","Type":"ContainerDied","Data":"2f655124e975981258127d10703872d21c94f5f1dedd7ae7c29ca637f5fbbd4b"}
Oct 30 13:19:54.321516 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:54.321487    4943 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-05942d/distinct-2-w1-05942d-kserve-855c7f7c5f-pwt79"
Oct 30 13:19:54.340544 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:54.340506    4943 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w0-031737-57d4df6b68-xr5s2"
Oct 30 13:19:54.344615 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:54.344593    4943 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-05942d/distinct-w1-05942d-kserve-5fb44dc664-mcbkc"
Oct 30 13:19:54.399966 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:54.399935    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d08eb359-cf9e-422c-a3a7-35fd173ab7b6-tls-certs\" (UniqueName: \"kubernetes.io/secret/d08eb359-cf9e-422c-a3a7-35fd173ab7b6-tls-certs\") pod \"d08eb359-cf9e-422c-a3a7-35fd173ab7b6\" (UID: \"d08eb359-cf9e-422c-a3a7-35fd173ab7b6\") "
Oct 30 13:19:54.400652 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:54.400630    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/3dac65dc-7a67-4fdb-840a-765235aaa5bf-tls-certs\" (UniqueName: \"kubernetes.io/secret/3dac65dc-7a67-4fdb-840a-765235aaa5bf-tls-certs\") pod \"3dac65dc-7a67-4fdb-840a-765235aaa5bf\" (UID: \"3dac65dc-7a67-4fdb-840a-765235aaa5bf\") "
Oct 30 13:19:54.400919 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:54.400898    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d08eb359-cf9e-422c-a3a7-35fd173ab7b6-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d08eb359-cf9e-422c-a3a7-35fd173ab7b6-tmp-dir\") pod \"d08eb359-cf9e-422c-a3a7-35fd173ab7b6\" (UID: \"d08eb359-cf9e-422c-a3a7-35fd173ab7b6\") "
Oct 30 13:19:54.401061 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:54.401043    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/9fc1504d-012b-4afd-bebf-7ac106d6990e-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9fc1504d-012b-4afd-bebf-7ac106d6990e-plugins-config-volume\") pod \"9fc1504d-012b-4afd-bebf-7ac106d6990e\" (UID: \"9fc1504d-012b-4afd-bebf-7ac106d6990e\") "
Oct 30 13:19:54.401253 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:54.401232    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d08eb359-cf9e-422c-a3a7-35fd173ab7b6-tmp-dir" pod "d08eb359-cf9e-422c-a3a7-35fd173ab7b6" (UID: "d08eb359-cf9e-422c-a3a7-35fd173ab7b6"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:19:54.401440 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:54.401421    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3dac65dc-7a67-4fdb-840a-765235aaa5bf-kube-api-access-jwtt4\" (UniqueName: \"kubernetes.io/projected/3dac65dc-7a67-4fdb-840a-765235aaa5bf-kube-api-access-jwtt4\") pod \"3dac65dc-7a67-4fdb-840a-765235aaa5bf\" (UID: \"3dac65dc-7a67-4fdb-840a-765235aaa5bf\") "
Oct 30 13:19:54.401571 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:54.401554    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d08eb359-cf9e-422c-a3a7-35fd173ab7b6-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d08eb359-cf9e-422c-a3a7-35fd173ab7b6-model-cache\") pod \"d08eb359-cf9e-422c-a3a7-35fd173ab7b6\" (UID: \"d08eb359-cf9e-422c-a3a7-35fd173ab7b6\") "
Oct 30 13:19:54.401699 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:54.401682    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d08eb359-cf9e-422c-a3a7-35fd173ab7b6-kube-api-access-xlflg\" (UniqueName: \"kubernetes.io/projected/d08eb359-cf9e-422c-a3a7-35fd173ab7b6-kube-api-access-xlflg\") pod \"d08eb359-cf9e-422c-a3a7-35fd173ab7b6\" (UID: \"d08eb359-cf9e-422c-a3a7-35fd173ab7b6\") "
Oct 30 13:19:54.401810 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:54.401794    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3dac65dc-7a67-4fdb-840a-765235aaa5bf-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3dac65dc-7a67-4fdb-840a-765235aaa5bf-tmp-dir\") pod \"3dac65dc-7a67-4fdb-840a-765235aaa5bf\" (UID: \"3dac65dc-7a67-4fdb-840a-765235aaa5bf\") "
Oct 30 13:19:54.401935 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:54.401918    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9fc1504d-012b-4afd-bebf-7ac106d6990e-kube-api-access-j79vn\" (UniqueName: \"kubernetes.io/projected/9fc1504d-012b-4afd-bebf-7ac106d6990e-kube-api-access-j79vn\") pod \"9fc1504d-012b-4afd-bebf-7ac106d6990e\" (UID: \"9fc1504d-012b-4afd-bebf-7ac106d6990e\") "
Oct 30 13:19:54.402337 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:54.401757    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9fc1504d-012b-4afd-bebf-7ac106d6990e-plugins-config-volume" pod "9fc1504d-012b-4afd-bebf-7ac106d6990e" (UID: "9fc1504d-012b-4afd-bebf-7ac106d6990e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 13:19:54.402471 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:54.402453    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3dac65dc-7a67-4fdb-840a-765235aaa5bf-dshm\" (UniqueName: \"kubernetes.io/empty-dir/3dac65dc-7a67-4fdb-840a-765235aaa5bf-dshm\") pod \"3dac65dc-7a67-4fdb-840a-765235aaa5bf\" (UID: \"3dac65dc-7a67-4fdb-840a-765235aaa5bf\") "
Oct 30 13:19:54.402640 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:54.402622    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d08eb359-cf9e-422c-a3a7-35fd173ab7b6-dshm\" (UniqueName: \"kubernetes.io/empty-dir/d08eb359-cf9e-422c-a3a7-35fd173ab7b6-dshm\") pod \"d08eb359-cf9e-422c-a3a7-35fd173ab7b6\" (UID: \"d08eb359-cf9e-422c-a3a7-35fd173ab7b6\") "
Oct 30 13:19:54.402737 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:54.402709    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d08eb359-cf9e-422c-a3a7-35fd173ab7b6-model-cache" pod "d08eb359-cf9e-422c-a3a7-35fd173ab7b6" (UID: "d08eb359-cf9e-422c-a3a7-35fd173ab7b6"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:19:54.402868 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:54.402850    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3dac65dc-7a67-4fdb-840a-765235aaa5bf-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3dac65dc-7a67-4fdb-840a-765235aaa5bf-model-cache\") pod \"3dac65dc-7a67-4fdb-840a-765235aaa5bf\" (UID: \"3dac65dc-7a67-4fdb-840a-765235aaa5bf\") "
Oct 30 13:19:54.403449 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:54.403016    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d08eb359-cf9e-422c-a3a7-35fd173ab7b6-tls-certs" pod "d08eb359-cf9e-422c-a3a7-35fd173ab7b6" (UID: "d08eb359-cf9e-422c-a3a7-35fd173ab7b6"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 13:19:54.403449 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:54.403170    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3dac65dc-7a67-4fdb-840a-765235aaa5bf-model-cache" pod "3dac65dc-7a67-4fdb-840a-765235aaa5bf" (UID: "3dac65dc-7a67-4fdb-840a-765235aaa5bf"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:19:54.403449 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:54.403356    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3dac65dc-7a67-4fdb-840a-765235aaa5bf-tmp-dir" pod "3dac65dc-7a67-4fdb-840a-765235aaa5bf" (UID: "3dac65dc-7a67-4fdb-840a-765235aaa5bf"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:19:54.404803 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:54.403719    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3dac65dc-7a67-4fdb-840a-765235aaa5bf-home" pod "3dac65dc-7a67-4fdb-840a-765235aaa5bf" (UID: "3dac65dc-7a67-4fdb-840a-765235aaa5bf"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:19:54.404803 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:54.403722    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3dac65dc-7a67-4fdb-840a-765235aaa5bf-home\" (UniqueName: \"kubernetes.io/empty-dir/3dac65dc-7a67-4fdb-840a-765235aaa5bf-home\") pod \"3dac65dc-7a67-4fdb-840a-765235aaa5bf\" (UID: \"3dac65dc-7a67-4fdb-840a-765235aaa5bf\") "
Oct 30 13:19:54.404803 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:54.403846    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d08eb359-cf9e-422c-a3a7-35fd173ab7b6-home\" (UniqueName: \"kubernetes.io/empty-dir/d08eb359-cf9e-422c-a3a7-35fd173ab7b6-home\") pod \"d08eb359-cf9e-422c-a3a7-35fd173ab7b6\" (UID: \"d08eb359-cf9e-422c-a3a7-35fd173ab7b6\") "
Oct 30 13:19:54.404803 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:54.404301    4943 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d08eb359-cf9e-422c-a3a7-35fd173ab7b6-tls-certs\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:19:54.404803 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:54.404323    4943 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d08eb359-cf9e-422c-a3a7-35fd173ab7b6-tmp-dir\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:19:54.404803 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:54.404337    4943 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9fc1504d-012b-4afd-bebf-7ac106d6990e-plugins-config-volume\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:19:54.404803 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:54.404351    4943 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d08eb359-cf9e-422c-a3a7-35fd173ab7b6-model-cache\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:19:54.404803 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:54.404366    4943 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3dac65dc-7a67-4fdb-840a-765235aaa5bf-tmp-dir\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:19:54.404803 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:54.404377    4943 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3dac65dc-7a67-4fdb-840a-765235aaa5bf-model-cache\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:19:54.404803 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:54.404389    4943 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3dac65dc-7a67-4fdb-840a-765235aaa5bf-home\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:19:54.404803 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:54.404776    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d08eb359-cf9e-422c-a3a7-35fd173ab7b6-home" pod "d08eb359-cf9e-422c-a3a7-35fd173ab7b6" (UID: "d08eb359-cf9e-422c-a3a7-35fd173ab7b6"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:19:54.405345 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:54.405143    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d08eb359-cf9e-422c-a3a7-35fd173ab7b6-dshm" pod "d08eb359-cf9e-422c-a3a7-35fd173ab7b6" (UID: "d08eb359-cf9e-422c-a3a7-35fd173ab7b6"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:19:54.406163 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:54.406117    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d08eb359-cf9e-422c-a3a7-35fd173ab7b6-kube-api-access-xlflg" pod "d08eb359-cf9e-422c-a3a7-35fd173ab7b6" (UID: "d08eb359-cf9e-422c-a3a7-35fd173ab7b6"). InnerVolumeSpecName "kube-api-access-xlflg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:19:54.406163 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:54.406135    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3dac65dc-7a67-4fdb-840a-765235aaa5bf-tls-certs" pod "3dac65dc-7a67-4fdb-840a-765235aaa5bf" (UID: "3dac65dc-7a67-4fdb-840a-765235aaa5bf"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 13:19:54.406443 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:54.406414    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3dac65dc-7a67-4fdb-840a-765235aaa5bf-kube-api-access-jwtt4" pod "3dac65dc-7a67-4fdb-840a-765235aaa5bf" (UID: "3dac65dc-7a67-4fdb-840a-765235aaa5bf"). InnerVolumeSpecName "kube-api-access-jwtt4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:19:54.407176 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:54.407153    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3dac65dc-7a67-4fdb-840a-765235aaa5bf-dshm" pod "3dac65dc-7a67-4fdb-840a-765235aaa5bf" (UID: "3dac65dc-7a67-4fdb-840a-765235aaa5bf"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:19:54.407465 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:54.407440    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9fc1504d-012b-4afd-bebf-7ac106d6990e-kube-api-access-j79vn" pod "9fc1504d-012b-4afd-bebf-7ac106d6990e" (UID: "9fc1504d-012b-4afd-bebf-7ac106d6990e"). InnerVolumeSpecName "kube-api-access-j79vn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:19:54.506373 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:54.506281    4943 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jwtt4\" (UniqueName: \"kubernetes.io/projected/3dac65dc-7a67-4fdb-840a-765235aaa5bf-kube-api-access-jwtt4\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:19:54.506373 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:54.506321    4943 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xlflg\" (UniqueName: \"kubernetes.io/projected/d08eb359-cf9e-422c-a3a7-35fd173ab7b6-kube-api-access-xlflg\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:19:54.506373 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:54.506336    4943 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j79vn\" (UniqueName: \"kubernetes.io/projected/9fc1504d-012b-4afd-bebf-7ac106d6990e-kube-api-access-j79vn\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:19:54.506373 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:54.506351    4943 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3dac65dc-7a67-4fdb-840a-765235aaa5bf-dshm\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:19:54.506373 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:54.506367    4943 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d08eb359-cf9e-422c-a3a7-35fd173ab7b6-dshm\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:19:54.506373 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:54.506379    4943 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d08eb359-cf9e-422c-a3a7-35fd173ab7b6-home\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:19:54.506801 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:54.506391    4943 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3dac65dc-7a67-4fdb-840a-765235aaa5bf-tls-certs\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:19:55.221368 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:55.221269    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-05942d/distinct-w1-05942d-kserve-5fb44dc664-mcbkc" event={"ID":"d08eb359-cf9e-422c-a3a7-35fd173ab7b6","Type":"ContainerDied","Data":"846de2f8ed53650a03f7f87e3d7e5f2a34f8f6ac48ecd17c6dea4eeec20f9cd4"}
Oct 30 13:19:55.221368 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:55.221299    4943 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-05942d/distinct-w1-05942d-kserve-5fb44dc664-mcbkc"
Oct 30 13:19:55.221368 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:55.221338    4943 scope.go:122] "RemoveContainer" containerID="5c0097f27d893027b33c144a75796ecbad447f17006917035307e622e520f2d7"
Oct 30 13:19:55.223594 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:55.223561    4943 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w0-031737-57d4df6b68-xr5s2"
Oct 30 13:19:55.223594 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:55.223581    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w0-031737-57d4df6b68-xr5s2" event={"ID":"9fc1504d-012b-4afd-bebf-7ac106d6990e","Type":"ContainerDied","Data":"44db439d26890bb6e3ee341c77ccc77aa448ca1aa0175b9e09265a1cc2694701"}
Oct 30 13:19:55.225593 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:55.225516    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-05942d/distinct-2-w1-05942d-kserve-855c7f7c5f-pwt79" event={"ID":"3dac65dc-7a67-4fdb-840a-765235aaa5bf","Type":"ContainerDied","Data":"26f0ed8486078bf57adf8b457209769b4addd1a3ccf9e15c445b9fbb3fdb64c7"}
Oct 30 13:19:55.225764 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:55.225531    4943 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-05942d/distinct-2-w1-05942d-kserve-855c7f7c5f-pwt79"
Oct 30 13:19:55.234075 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:55.234020    4943 scope.go:122] "RemoveContainer" containerID="2f655124e975981258127d10703872d21c94f5f1dedd7ae7c29ca637f5fbbd4b"
Oct 30 13:19:55.244414 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:55.244389    4943 scope.go:122] "RemoveContainer" containerID="632ff98e6b4cf4c86974d5cbbb1124f5f9e7eeacca5c2759c012e3e910264255"
Oct 30 13:19:55.277337 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:55.277301    4943 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-05942d/distinct-2-w1-05942d-kserve-855c7f7c5f-pwt79" podUID="3dac65dc-7a67-4fdb-840a-765235aaa5bf" containerName="main" probeResult="failure" output="Get \"https://10.132.0.95:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:19:55.291722 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:55.291693    4943 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-031737-57d4df6b68-xr5s2"]
Oct 30 13:19:55.296028 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:55.296005    4943 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-031737-57d4df6b68-xr5s2"]
Oct 30 13:19:55.335725 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:55.335689    4943 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w1-05942d/distinct-w1-05942d-kserve-5fb44dc664-mcbkc"]
Oct 30 13:19:55.349574 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:55.349544    4943 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w1-05942d/distinct-w1-05942d-kserve-5fb44dc664-mcbkc"]
Oct 30 13:19:55.389748 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:55.389718    4943 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w1-05942d/distinct-2-w1-05942d-kserve-855c7f7c5f-pwt79"]
Oct 30 13:19:55.405817 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:55.405790    4943 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w1-05942d/distinct-2-w1-05942d-kserve-855c7f7c5f-pwt79"]
Oct 30 13:19:56.350899 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:56.350858    4943 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="3dac65dc-7a67-4fdb-840a-765235aaa5bf" path="/var/lib/kubelet/pods/3dac65dc-7a67-4fdb-840a-765235aaa5bf/volumes"
Oct 30 13:19:56.351379 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:56.351362    4943 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9fc1504d-012b-4afd-bebf-7ac106d6990e" path="/var/lib/kubelet/pods/9fc1504d-012b-4afd-bebf-7ac106d6990e/volumes"
Oct 30 13:19:56.351831 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:56.351815    4943 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d08eb359-cf9e-422c-a3a7-35fd173ab7b6" path="/var/lib/kubelet/pods/d08eb359-cf9e-422c-a3a7-35fd173ab7b6/volumes"
Oct 30 13:19:59.648759 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:19:59.648720    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-hdmrm"]
Oct 30 13:20:00.873317 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:00.873249    4943 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w1-05942d-zf6sc"]
Oct 30 13:20:00.879433 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:00.879399    4943 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w1-05942d-zf6sc"]
Oct 30 13:20:01.000312 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:00.999425    4943 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w3-d0a18c-b6l49"]
Oct 30 13:20:01.000312 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:01.000180    4943 cpu_manager.go:414] "containerMap: removing container" podUID="d08eb359-cf9e-422c-a3a7-35fd173ab7b6" containerName="main"
Oct 30 13:20:01.000312 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:01.000202    4943 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d08eb359-cf9e-422c-a3a7-35fd173ab7b6" containerName="main"
Oct 30 13:20:01.000312 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:01.000218    4943 cpu_manager.go:414] "containerMap: removing container" podUID="5a018183-4b6a-4d06-a087-34c2f5168b46" containerName="revoke-keys"
Oct 30 13:20:01.000312 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:01.000223    4943 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5a018183-4b6a-4d06-a087-34c2f5168b46" containerName="revoke-keys"
Oct 30 13:20:01.000312 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:01.000235    4943 cpu_manager.go:414] "containerMap: removing container" podUID="3dac65dc-7a67-4fdb-840a-765235aaa5bf" containerName="main"
Oct 30 13:20:01.000312 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:01.000241    4943 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3dac65dc-7a67-4fdb-840a-765235aaa5bf" containerName="main"
Oct 30 13:20:01.000312 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:01.000269    4943 cpu_manager.go:414] "containerMap: removing container" podUID="8c4e0447-d0df-4536-bea2-00f13e370c5b" containerName="istio-proxy"
Oct 30 13:20:01.000312 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:01.000277    4943 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8c4e0447-d0df-4536-bea2-00f13e370c5b" containerName="istio-proxy"
Oct 30 13:20:01.000312 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:01.000312    4943 cpu_manager.go:414] "containerMap: removing container" podUID="3eba0196-49e5-42ea-8192-ccaac19b1443" containerName="istio-proxy"
Oct 30 13:20:01.000312 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:01.000329    4943 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3eba0196-49e5-42ea-8192-ccaac19b1443" containerName="istio-proxy"
Oct 30 13:20:01.000312 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:01.000338    4943 cpu_manager.go:414] "containerMap: removing container" podUID="1b91fb8f-779b-440c-b6a7-d2b89bf020a1" containerName="revoke-keys"
Oct 30 13:20:01.000312 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:01.000344    4943 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1b91fb8f-779b-440c-b6a7-d2b89bf020a1" containerName="revoke-keys"
Oct 30 13:20:01.001116 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:01.000365    4943 cpu_manager.go:414] "containerMap: removing container" podUID="126cf7ee-cae5-4afa-82f5-f36102a35b59" containerName="istio-proxy"
Oct 30 13:20:01.001116 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:01.000374    4943 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="126cf7ee-cae5-4afa-82f5-f36102a35b59" containerName="istio-proxy"
Oct 30 13:20:01.001116 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:01.000383    4943 cpu_manager.go:414] "containerMap: removing container" podUID="9fc1504d-012b-4afd-bebf-7ac106d6990e" containerName="payload-processing"
Oct 30 13:20:01.001116 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:01.000388    4943 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9fc1504d-012b-4afd-bebf-7ac106d6990e" containerName="payload-processing"
Oct 30 13:20:01.001116 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:01.000401    4943 cpu_manager.go:414] "containerMap: removing container" podUID="558c9168-9f89-44f8-845b-74fe5345386c" containerName="istio-proxy"
Oct 30 13:20:01.001116 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:01.000405    4943 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="558c9168-9f89-44f8-845b-74fe5345386c" containerName="istio-proxy"
Oct 30 13:20:01.001116 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:01.000523    4943 memory_manager.go:365] "RemoveStaleState removing state" podUID="3eba0196-49e5-42ea-8192-ccaac19b1443" containerName="istio-proxy"
Oct 30 13:20:01.001116 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:01.000537    4943 memory_manager.go:365] "RemoveStaleState removing state" podUID="1b91fb8f-779b-440c-b6a7-d2b89bf020a1" containerName="revoke-keys"
Oct 30 13:20:01.001116 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:01.000549    4943 memory_manager.go:365] "RemoveStaleState removing state" podUID="126cf7ee-cae5-4afa-82f5-f36102a35b59" containerName="istio-proxy"
Oct 30 13:20:01.001116 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:01.000558    4943 memory_manager.go:365] "RemoveStaleState removing state" podUID="9fc1504d-012b-4afd-bebf-7ac106d6990e" containerName="payload-processing"
Oct 30 13:20:01.001116 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:01.000570    4943 memory_manager.go:365] "RemoveStaleState removing state" podUID="8c4e0447-d0df-4536-bea2-00f13e370c5b" containerName="istio-proxy"
Oct 30 13:20:01.001116 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:01.000578    4943 memory_manager.go:365] "RemoveStaleState removing state" podUID="558c9168-9f89-44f8-845b-74fe5345386c" containerName="istio-proxy"
Oct 30 13:20:01.001116 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:01.000587    4943 memory_manager.go:365] "RemoveStaleState removing state" podUID="d08eb359-cf9e-422c-a3a7-35fd173ab7b6" containerName="main"
Oct 30 13:20:01.001116 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:01.000596    4943 memory_manager.go:365] "RemoveStaleState removing state" podUID="5a018183-4b6a-4d06-a087-34c2f5168b46" containerName="revoke-keys"
Oct 30 13:20:01.001116 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:01.000605    4943 memory_manager.go:365] "RemoveStaleState removing state" podUID="3dac65dc-7a67-4fdb-840a-765235aaa5bf" containerName="main"
Oct 30 13:20:01.001116 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:01.000615    4943 memory_manager.go:365] "RemoveStaleState removing state" podUID="8c4e0447-d0df-4536-bea2-00f13e370c5b" containerName="istio-proxy"
Oct 30 13:20:01.006501 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:01.006474    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w3-d0a18c-b6l49"
Oct 30 13:20:01.051285 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:01.030646    4943 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-p9z57\""
Oct 30 13:20:01.070635 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:01.067515    4943 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Oct 30 13:20:01.070635 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:01.067634    4943 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Oct 30 13:20:01.070635 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:01.067695    4943 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 13:20:01.167339 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:01.159795    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w3-d0a18c-b6l49"]
Oct 30 13:20:01.224278 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:01.191369    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/08f92795-8d3e-465d-8571-1b4f5a0133b3-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-worker-w3-d0a18c-b6l49\" (UID: \"08f92795-8d3e-465d-8571-1b4f5a0133b3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w3-d0a18c-b6l49"
Oct 30 13:20:01.224278 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:01.191614    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j2dgm\" (UniqueName: \"kubernetes.io/projected/08f92795-8d3e-465d-8571-1b4f5a0133b3-kube-api-access-j2dgm\") pod \"maas-api-revoke-keys-e2e-worker-w3-d0a18c-b6l49\" (UID: \"08f92795-8d3e-465d-8571-1b4f5a0133b3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w3-d0a18c-b6l49"
Oct 30 13:20:01.224278 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:01.191678    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/08f92795-8d3e-465d-8571-1b4f5a0133b3-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-worker-w3-d0a18c-b6l49\" (UID: \"08f92795-8d3e-465d-8571-1b4f5a0133b3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w3-d0a18c-b6l49"
Oct 30 13:20:01.341330 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:01.292610    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/08f92795-8d3e-465d-8571-1b4f5a0133b3-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-worker-w3-d0a18c-b6l49\" (UID: \"08f92795-8d3e-465d-8571-1b4f5a0133b3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w3-d0a18c-b6l49"
Oct 30 13:20:01.341330 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:01.292751    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/08f92795-8d3e-465d-8571-1b4f5a0133b3-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-worker-w3-d0a18c-b6l49\" (UID: \"08f92795-8d3e-465d-8571-1b4f5a0133b3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w3-d0a18c-b6l49"
Oct 30 13:20:01.341330 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:01.292897    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j2dgm\" (UniqueName: \"kubernetes.io/projected/08f92795-8d3e-465d-8571-1b4f5a0133b3-kube-api-access-j2dgm\") pod \"maas-api-revoke-keys-e2e-worker-w3-d0a18c-b6l49\" (UID: \"08f92795-8d3e-465d-8571-1b4f5a0133b3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w3-d0a18c-b6l49"
Oct 30 13:20:01.341330 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:01.294024    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/08f92795-8d3e-465d-8571-1b4f5a0133b3-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-worker-w3-d0a18c-b6l49\" (UID: \"08f92795-8d3e-465d-8571-1b4f5a0133b3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w3-d0a18c-b6l49"
Oct 30 13:20:01.483974 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:01.483936    4943 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w1-05942d-7d456fbd84-mvdr9" podUID="7607a99a-237c-4d0c-b5d0-5168ec6a6e3f" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 13:20:01.519079 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:01.519044    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/08f92795-8d3e-465d-8571-1b4f5a0133b3-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-worker-w3-d0a18c-b6l49\" (UID: \"08f92795-8d3e-465d-8571-1b4f5a0133b3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w3-d0a18c-b6l49"
Oct 30 13:20:01.521214 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:01.521115    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j2dgm\" (UniqueName: \"kubernetes.io/projected/08f92795-8d3e-465d-8571-1b4f5a0133b3-kube-api-access-j2dgm\") pod \"maas-api-revoke-keys-e2e-worker-w3-d0a18c-b6l49\" (UID: \"08f92795-8d3e-465d-8571-1b4f5a0133b3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w3-d0a18c-b6l49"
Oct 30 13:20:01.690743 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:01.690705    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w3-d0a18c-b6l49"
Oct 30 13:20:02.056285 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:02.020869    4943 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-031737-rd656"]
Oct 30 13:20:02.231281 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:02.231065    4943 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-ab3176/simulator-w6-ab3176-kserve-6d9b64549f-dc8xz" podUID="da306792-4f80-4df4-aa44-82991ab856e4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.69:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:20:02.512342 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:02.511377    4943 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-d0a18c/distinct-w3-d0a18c-kserve-86479f6bbf-5zcqq" podUID="63dace4c-7832-47dd-84ab-2c91483a5563" containerName="main" probeResult="failure" output="Get \"https://10.132.0.89:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:20:02.612291 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:02.579309    4943 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="03266586-4be7-42d3-85e5-e607b3754354" path="/var/lib/kubelet/pods/03266586-4be7-42d3-85e5-e607b3754354/volumes"
Oct 30 13:20:02.631647 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:02.631594    4943 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-031737-rd656"]
Oct 30 13:20:02.653576 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:02.653542    4943 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-ljkqn"]
Oct 30 13:20:02.654737 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:02.654709    4943 cpu_manager.go:414] "containerMap: removing container" podUID="8c4e0447-d0df-4536-bea2-00f13e370c5b" containerName="istio-proxy"
Oct 30 13:20:02.654881 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:02.654866    4943 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8c4e0447-d0df-4536-bea2-00f13e370c5b" containerName="istio-proxy"
Oct 30 13:20:02.661144 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:02.661123    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-ljkqn"
Oct 30 13:20:02.685282 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:02.668432    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f0ab36b0-4ba8-41da-8fc2-6a70d1082599-istio-envoy\") pod \"e2e-worker-w2-f76c09-openshift-default-65d5845b67-ljkqn\" (UID: \"f0ab36b0-4ba8-41da-8fc2-6a70d1082599\") " pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-ljkqn"
Oct 30 13:20:02.687779 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:02.687750    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f0ab36b0-4ba8-41da-8fc2-6a70d1082599-istio-token\") pod \"e2e-worker-w2-f76c09-openshift-default-65d5845b67-ljkqn\" (UID: \"f0ab36b0-4ba8-41da-8fc2-6a70d1082599\") " pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-ljkqn"
Oct 30 13:20:02.687953 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:02.687932    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f0ab36b0-4ba8-41da-8fc2-6a70d1082599-istio-data\") pod \"e2e-worker-w2-f76c09-openshift-default-65d5845b67-ljkqn\" (UID: \"f0ab36b0-4ba8-41da-8fc2-6a70d1082599\") " pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-ljkqn"
Oct 30 13:20:02.688076 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:02.688059    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f0ab36b0-4ba8-41da-8fc2-6a70d1082599-istio-podinfo\") pod \"e2e-worker-w2-f76c09-openshift-default-65d5845b67-ljkqn\" (UID: \"f0ab36b0-4ba8-41da-8fc2-6a70d1082599\") " pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-ljkqn"
Oct 30 13:20:02.688221 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:02.688203    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f0ab36b0-4ba8-41da-8fc2-6a70d1082599-istiod-ca-cert\") pod \"e2e-worker-w2-f76c09-openshift-default-65d5845b67-ljkqn\" (UID: \"f0ab36b0-4ba8-41da-8fc2-6a70d1082599\") " pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-ljkqn"
Oct 30 13:20:02.688377 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:02.688357    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f0ab36b0-4ba8-41da-8fc2-6a70d1082599-credential-socket\") pod \"e2e-worker-w2-f76c09-openshift-default-65d5845b67-ljkqn\" (UID: \"f0ab36b0-4ba8-41da-8fc2-6a70d1082599\") " pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-ljkqn"
Oct 30 13:20:02.688579 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:02.688561    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f0ab36b0-4ba8-41da-8fc2-6a70d1082599-workload-socket\") pod \"e2e-worker-w2-f76c09-openshift-default-65d5845b67-ljkqn\" (UID: \"f0ab36b0-4ba8-41da-8fc2-6a70d1082599\") " pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-ljkqn"
Oct 30 13:20:02.688696 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:02.688680    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k2cbg\" (UniqueName: \"kubernetes.io/projected/f0ab36b0-4ba8-41da-8fc2-6a70d1082599-kube-api-access-k2cbg\") pod \"e2e-worker-w2-f76c09-openshift-default-65d5845b67-ljkqn\" (UID: \"f0ab36b0-4ba8-41da-8fc2-6a70d1082599\") " pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-ljkqn"
Oct 30 13:20:02.688848 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:02.688831    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f0ab36b0-4ba8-41da-8fc2-6a70d1082599-workload-certs\") pod \"e2e-worker-w2-f76c09-openshift-default-65d5845b67-ljkqn\" (UID: \"f0ab36b0-4ba8-41da-8fc2-6a70d1082599\") " pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-ljkqn"
Oct 30 13:20:02.749615 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:02.740321    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-ljkqn"]
Oct 30 13:20:02.754434 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:02.754403    4943 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-pph7x"]
Oct 30 13:20:02.786842 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:02.786753    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-pph7x"
Oct 30 13:20:02.797656 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:02.791054    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f0ab36b0-4ba8-41da-8fc2-6a70d1082599-istio-envoy\") pod \"e2e-worker-w2-f76c09-openshift-default-65d5845b67-ljkqn\" (UID: \"f0ab36b0-4ba8-41da-8fc2-6a70d1082599\") " pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-ljkqn"
Oct 30 13:20:02.797656 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:02.791140    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f0ab36b0-4ba8-41da-8fc2-6a70d1082599-istio-token\") pod \"e2e-worker-w2-f76c09-openshift-default-65d5845b67-ljkqn\" (UID: \"f0ab36b0-4ba8-41da-8fc2-6a70d1082599\") " pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-ljkqn"
Oct 30 13:20:02.797656 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:02.791176    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f0ab36b0-4ba8-41da-8fc2-6a70d1082599-istio-data\") pod \"e2e-worker-w2-f76c09-openshift-default-65d5845b67-ljkqn\" (UID: \"f0ab36b0-4ba8-41da-8fc2-6a70d1082599\") " pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-ljkqn"
Oct 30 13:20:02.797656 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:02.791206    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f0ab36b0-4ba8-41da-8fc2-6a70d1082599-istio-podinfo\") pod \"e2e-worker-w2-f76c09-openshift-default-65d5845b67-ljkqn\" (UID: \"f0ab36b0-4ba8-41da-8fc2-6a70d1082599\") " pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-ljkqn"
Oct 30 13:20:02.797656 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:02.791234    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f0ab36b0-4ba8-41da-8fc2-6a70d1082599-istiod-ca-cert\") pod \"e2e-worker-w2-f76c09-openshift-default-65d5845b67-ljkqn\" (UID: \"f0ab36b0-4ba8-41da-8fc2-6a70d1082599\") " pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-ljkqn"
Oct 30 13:20:02.797656 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:02.791283    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f0ab36b0-4ba8-41da-8fc2-6a70d1082599-credential-socket\") pod \"e2e-worker-w2-f76c09-openshift-default-65d5845b67-ljkqn\" (UID: \"f0ab36b0-4ba8-41da-8fc2-6a70d1082599\") " pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-ljkqn"
Oct 30 13:20:02.797656 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:02.791355    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f0ab36b0-4ba8-41da-8fc2-6a70d1082599-workload-socket\") pod \"e2e-worker-w2-f76c09-openshift-default-65d5845b67-ljkqn\" (UID: \"f0ab36b0-4ba8-41da-8fc2-6a70d1082599\") " pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-ljkqn"
Oct 30 13:20:02.797656 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:02.791378    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k2cbg\" (UniqueName: \"kubernetes.io/projected/f0ab36b0-4ba8-41da-8fc2-6a70d1082599-kube-api-access-k2cbg\") pod \"e2e-worker-w2-f76c09-openshift-default-65d5845b67-ljkqn\" (UID: \"f0ab36b0-4ba8-41da-8fc2-6a70d1082599\") " pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-ljkqn"
Oct 30 13:20:02.797656 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:02.791434    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f0ab36b0-4ba8-41da-8fc2-6a70d1082599-workload-certs\") pod \"e2e-worker-w2-f76c09-openshift-default-65d5845b67-ljkqn\" (UID: \"f0ab36b0-4ba8-41da-8fc2-6a70d1082599\") " pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-ljkqn"
Oct 30 13:20:02.797656 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:02.791963    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f0ab36b0-4ba8-41da-8fc2-6a70d1082599-workload-certs\") pod \"e2e-worker-w2-f76c09-openshift-default-65d5845b67-ljkqn\" (UID: \"f0ab36b0-4ba8-41da-8fc2-6a70d1082599\") " pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-ljkqn"
Oct 30 13:20:02.818238 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:02.800564    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f0ab36b0-4ba8-41da-8fc2-6a70d1082599-istiod-ca-cert\") pod \"e2e-worker-w2-f76c09-openshift-default-65d5845b67-ljkqn\" (UID: \"f0ab36b0-4ba8-41da-8fc2-6a70d1082599\") " pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-ljkqn"
Oct 30 13:20:02.818238 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:02.801179    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f0ab36b0-4ba8-41da-8fc2-6a70d1082599-istio-data\") pod \"e2e-worker-w2-f76c09-openshift-default-65d5845b67-ljkqn\" (UID: \"f0ab36b0-4ba8-41da-8fc2-6a70d1082599\") " pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-ljkqn"
Oct 30 13:20:02.818238 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:02.802580    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f0ab36b0-4ba8-41da-8fc2-6a70d1082599-workload-socket\") pod \"e2e-worker-w2-f76c09-openshift-default-65d5845b67-ljkqn\" (UID: \"f0ab36b0-4ba8-41da-8fc2-6a70d1082599\") " pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-ljkqn"
Oct 30 13:20:02.818238 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:02.802863    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f0ab36b0-4ba8-41da-8fc2-6a70d1082599-credential-socket\") pod \"e2e-worker-w2-f76c09-openshift-default-65d5845b67-ljkqn\" (UID: \"f0ab36b0-4ba8-41da-8fc2-6a70d1082599\") " pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-ljkqn"
Oct 30 13:20:02.914046 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:02.895396    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f0ab36b0-4ba8-41da-8fc2-6a70d1082599-istio-podinfo\") pod \"e2e-worker-w2-f76c09-openshift-default-65d5845b67-ljkqn\" (UID: \"f0ab36b0-4ba8-41da-8fc2-6a70d1082599\") " pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-ljkqn"
Oct 30 13:20:02.914046 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:02.896429    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bd83c13a-8f22-4ada-a894-90cd49e87782-credential-socket\") pod \"e2e-worker-w2-f76c09-openshift-default-65d5845b67-pph7x\" (UID: \"bd83c13a-8f22-4ada-a894-90cd49e87782\") " pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-pph7x"
Oct 30 13:20:02.914046 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:02.896489    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bd83c13a-8f22-4ada-a894-90cd49e87782-istio-data\") pod \"e2e-worker-w2-f76c09-openshift-default-65d5845b67-pph7x\" (UID: \"bd83c13a-8f22-4ada-a894-90cd49e87782\") " pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-pph7x"
Oct 30 13:20:02.914046 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:02.896516    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bd83c13a-8f22-4ada-a894-90cd49e87782-istio-token\") pod \"e2e-worker-w2-f76c09-openshift-default-65d5845b67-pph7x\" (UID: \"bd83c13a-8f22-4ada-a894-90cd49e87782\") " pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-pph7x"
Oct 30 13:20:02.914046 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:02.896545    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bd83c13a-8f22-4ada-a894-90cd49e87782-istiod-ca-cert\") pod \"e2e-worker-w2-f76c09-openshift-default-65d5845b67-pph7x\" (UID: \"bd83c13a-8f22-4ada-a894-90cd49e87782\") " pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-pph7x"
Oct 30 13:20:02.914046 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:02.896649    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bd83c13a-8f22-4ada-a894-90cd49e87782-workload-certs\") pod \"e2e-worker-w2-f76c09-openshift-default-65d5845b67-pph7x\" (UID: \"bd83c13a-8f22-4ada-a894-90cd49e87782\") " pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-pph7x"
Oct 30 13:20:02.914046 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:02.896692    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xlprp\" (UniqueName: \"kubernetes.io/projected/bd83c13a-8f22-4ada-a894-90cd49e87782-kube-api-access-xlprp\") pod \"e2e-worker-w2-f76c09-openshift-default-65d5845b67-pph7x\" (UID: \"bd83c13a-8f22-4ada-a894-90cd49e87782\") " pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-pph7x"
Oct 30 13:20:02.914046 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:02.896733    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bd83c13a-8f22-4ada-a894-90cd49e87782-workload-socket\") pod \"e2e-worker-w2-f76c09-openshift-default-65d5845b67-pph7x\" (UID: \"bd83c13a-8f22-4ada-a894-90cd49e87782\") " pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-pph7x"
Oct 30 13:20:02.914046 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:02.896795    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bd83c13a-8f22-4ada-a894-90cd49e87782-istio-podinfo\") pod \"e2e-worker-w2-f76c09-openshift-default-65d5845b67-pph7x\" (UID: \"bd83c13a-8f22-4ada-a894-90cd49e87782\") " pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-pph7x"
Oct 30 13:20:02.914046 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:02.896826    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bd83c13a-8f22-4ada-a894-90cd49e87782-istio-envoy\") pod \"e2e-worker-w2-f76c09-openshift-default-65d5845b67-pph7x\" (UID: \"bd83c13a-8f22-4ada-a894-90cd49e87782\") " pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-pph7x"
Oct 30 13:20:02.914046 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:02.898952    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f0ab36b0-4ba8-41da-8fc2-6a70d1082599-istio-token\") pod \"e2e-worker-w2-f76c09-openshift-default-65d5845b67-ljkqn\" (UID: \"f0ab36b0-4ba8-41da-8fc2-6a70d1082599\") " pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-ljkqn"
Oct 30 13:20:02.914046 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:02.907394    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-pph7x"]
Oct 30 13:20:02.914046 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:02.911462    4943 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-d0a18c/distinct-2-w3-d0a18c-kserve-85f8ff4846-rc44r" podUID="d65e47d2-4461-406a-9b71-bb3fac4262f0" containerName="main" probeResult="failure" output="Get \"https://10.132.0.93:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:20:02.919288 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:02.915439    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f0ab36b0-4ba8-41da-8fc2-6a70d1082599-istio-envoy\") pod \"e2e-worker-w2-f76c09-openshift-default-65d5845b67-ljkqn\" (UID: \"f0ab36b0-4ba8-41da-8fc2-6a70d1082599\") " pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-ljkqn"
Oct 30 13:20:02.967285 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:02.950773    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k2cbg\" (UniqueName: \"kubernetes.io/projected/f0ab36b0-4ba8-41da-8fc2-6a70d1082599-kube-api-access-k2cbg\") pod \"e2e-worker-w2-f76c09-openshift-default-65d5845b67-ljkqn\" (UID: \"f0ab36b0-4ba8-41da-8fc2-6a70d1082599\") " pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-ljkqn"
Oct 30 13:20:03.056397 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:03.017066    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xlprp\" (UniqueName: \"kubernetes.io/projected/bd83c13a-8f22-4ada-a894-90cd49e87782-kube-api-access-xlprp\") pod \"e2e-worker-w2-f76c09-openshift-default-65d5845b67-pph7x\" (UID: \"bd83c13a-8f22-4ada-a894-90cd49e87782\") " pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-pph7x"
Oct 30 13:20:03.056397 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:03.017168    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bd83c13a-8f22-4ada-a894-90cd49e87782-workload-socket\") pod \"e2e-worker-w2-f76c09-openshift-default-65d5845b67-pph7x\" (UID: \"bd83c13a-8f22-4ada-a894-90cd49e87782\") " pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-pph7x"
Oct 30 13:20:03.056397 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:03.017232    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bd83c13a-8f22-4ada-a894-90cd49e87782-istio-podinfo\") pod \"e2e-worker-w2-f76c09-openshift-default-65d5845b67-pph7x\" (UID: \"bd83c13a-8f22-4ada-a894-90cd49e87782\") " pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-pph7x"
Oct 30 13:20:03.056397 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:03.017282    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bd83c13a-8f22-4ada-a894-90cd49e87782-istio-envoy\") pod \"e2e-worker-w2-f76c09-openshift-default-65d5845b67-pph7x\" (UID: \"bd83c13a-8f22-4ada-a894-90cd49e87782\") " pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-pph7x"
Oct 30 13:20:03.056397 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:03.017429    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bd83c13a-8f22-4ada-a894-90cd49e87782-credential-socket\") pod \"e2e-worker-w2-f76c09-openshift-default-65d5845b67-pph7x\" (UID: \"bd83c13a-8f22-4ada-a894-90cd49e87782\") " pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-pph7x"
Oct 30 13:20:03.056397 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:03.017472    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bd83c13a-8f22-4ada-a894-90cd49e87782-istio-data\") pod \"e2e-worker-w2-f76c09-openshift-default-65d5845b67-pph7x\" (UID: \"bd83c13a-8f22-4ada-a894-90cd49e87782\") " pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-pph7x"
Oct 30 13:20:03.056397 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:03.017495    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bd83c13a-8f22-4ada-a894-90cd49e87782-istio-token\") pod \"e2e-worker-w2-f76c09-openshift-default-65d5845b67-pph7x\" (UID: \"bd83c13a-8f22-4ada-a894-90cd49e87782\") " pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-pph7x"
Oct 30 13:20:03.056397 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:03.017525    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bd83c13a-8f22-4ada-a894-90cd49e87782-istiod-ca-cert\") pod \"e2e-worker-w2-f76c09-openshift-default-65d5845b67-pph7x\" (UID: \"bd83c13a-8f22-4ada-a894-90cd49e87782\") " pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-pph7x"
Oct 30 13:20:03.056397 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:03.017619    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bd83c13a-8f22-4ada-a894-90cd49e87782-workload-certs\") pod \"e2e-worker-w2-f76c09-openshift-default-65d5845b67-pph7x\" (UID: \"bd83c13a-8f22-4ada-a894-90cd49e87782\") " pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-pph7x"
Oct 30 13:20:03.056397 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:03.018104    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bd83c13a-8f22-4ada-a894-90cd49e87782-workload-certs\") pod \"e2e-worker-w2-f76c09-openshift-default-65d5845b67-pph7x\" (UID: \"bd83c13a-8f22-4ada-a894-90cd49e87782\") " pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-pph7x"
Oct 30 13:20:03.056397 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:03.018710    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bd83c13a-8f22-4ada-a894-90cd49e87782-workload-socket\") pod \"e2e-worker-w2-f76c09-openshift-default-65d5845b67-pph7x\" (UID: \"bd83c13a-8f22-4ada-a894-90cd49e87782\") " pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-pph7x"
Oct 30 13:20:03.075312 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:03.062956    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bd83c13a-8f22-4ada-a894-90cd49e87782-istio-podinfo\") pod \"e2e-worker-w2-f76c09-openshift-default-65d5845b67-pph7x\" (UID: \"bd83c13a-8f22-4ada-a894-90cd49e87782\") " pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-pph7x"
Oct 30 13:20:03.210222 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:03.076772    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bd83c13a-8f22-4ada-a894-90cd49e87782-credential-socket\") pod \"e2e-worker-w2-f76c09-openshift-default-65d5845b67-pph7x\" (UID: \"bd83c13a-8f22-4ada-a894-90cd49e87782\") " pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-pph7x"
Oct 30 13:20:03.210222 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:03.077059    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bd83c13a-8f22-4ada-a894-90cd49e87782-istio-data\") pod \"e2e-worker-w2-f76c09-openshift-default-65d5845b67-pph7x\" (UID: \"bd83c13a-8f22-4ada-a894-90cd49e87782\") " pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-pph7x"
Oct 30 13:20:03.210222 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:03.077976    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bd83c13a-8f22-4ada-a894-90cd49e87782-istiod-ca-cert\") pod \"e2e-worker-w2-f76c09-openshift-default-65d5845b67-pph7x\" (UID: \"bd83c13a-8f22-4ada-a894-90cd49e87782\") " pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-pph7x"
Oct 30 13:20:03.210222 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:03.099086    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-ljkqn"
Oct 30 13:20:03.211342 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:03.210912    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bd83c13a-8f22-4ada-a894-90cd49e87782-istio-envoy\") pod \"e2e-worker-w2-f76c09-openshift-default-65d5845b67-pph7x\" (UID: \"bd83c13a-8f22-4ada-a894-90cd49e87782\") " pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-pph7x"
Oct 30 13:20:03.256556 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:03.256346    4943 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-ab3176/simulator-w6-ab3176-kserve-6d9b64549f-dc8xz" podUID="da306792-4f80-4df4-aa44-82991ab856e4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.69:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:20:03.257292 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:03.256799    4943 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-ab3176/simulator-w6-ab3176-kserve-6d9b64549f-dc8xz"
Oct 30 13:20:03.392282 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:03.323128    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xlprp\" (UniqueName: \"kubernetes.io/projected/bd83c13a-8f22-4ada-a894-90cd49e87782-kube-api-access-xlprp\") pod \"e2e-worker-w2-f76c09-openshift-default-65d5845b67-pph7x\" (UID: \"bd83c13a-8f22-4ada-a894-90cd49e87782\") " pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-pph7x"
Oct 30 13:20:03.512664 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:03.512621    4943 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-d0a18c/distinct-w3-d0a18c-kserve-86479f6bbf-5zcqq" podUID="63dace4c-7832-47dd-84ab-2c91483a5563" containerName="main" probeResult="failure" output="Get \"https://10.132.0.89:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:20:03.513525 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:03.513501    4943 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-d0a18c/distinct-w3-d0a18c-kserve-86479f6bbf-5zcqq"
Oct 30 13:20:03.549129 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:03.549082    4943 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w3-d0a18c/distinct-2-w3-d0a18c-kserve-85f8ff4846-rc44r" podUID="d65e47d2-4461-406a-9b71-bb3fac4262f0" containerName="main" probeResult="failure" output="Get \"https://10.132.0.93:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:20:03.727546 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:03.727508    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bd83c13a-8f22-4ada-a894-90cd49e87782-istio-token\") pod \"e2e-worker-w2-f76c09-openshift-default-65d5845b67-pph7x\" (UID: \"bd83c13a-8f22-4ada-a894-90cd49e87782\") " pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-pph7x"
Oct 30 13:20:03.803465 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:03.803420    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-pph7x"
Oct 30 13:20:03.948669 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:03.948627    4943 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-d0a18c/distinct-2-w3-d0a18c-kserve-85f8ff4846-rc44r" podUID="d65e47d2-4461-406a-9b71-bb3fac4262f0" containerName="main" probeResult="failure" output="Get \"https://10.132.0.93:8000/ready\": context deadline exceeded"
Oct 30 13:20:03.950602 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:03.950578    4943 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-d0a18c/distinct-2-w3-d0a18c-kserve-85f8ff4846-rc44r"
Oct 30 13:20:04.261663 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:04.261597    4943 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-ab3176/simulator-w6-ab3176-kserve-6d9b64549f-dc8xz" podUID="da306792-4f80-4df4-aa44-82991ab856e4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.69:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:20:04.276315 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:04.271993    4943 patch_prober.go:29] interesting pod/e2e-worker-w3-d0a18c-openshift-default-547d57db97-sfbdb container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.100:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 13:20:04.276315 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:04.272073    4943 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-sfbdb" podUID="4a720f44-f6a3-48b5-8ba4-04b3214486eb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.100:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:20:04.560578 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:04.539283    4943 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-d0a18c/distinct-w3-d0a18c-kserve-86479f6bbf-5zcqq" podUID="63dace4c-7832-47dd-84ab-2c91483a5563" containerName="main" probeResult="failure" output="Get \"https://10.132.0.89:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:20:04.612285 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:04.609206    4943 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="963b4f75-ac69-43a1-9925-35acd0c9de8a" path="/var/lib/kubelet/pods/963b4f75-ac69-43a1-9925-35acd0c9de8a/volumes"
Oct 30 13:20:04.724461 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:04.724424    4943 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w2-f76c09/distinct-2-w2-f76c09-kserve-78446dcc4c-7ljvf"]
Oct 30 13:20:05.020462 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:04.954326    4943 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-d0a18c/distinct-2-w3-d0a18c-kserve-85f8ff4846-rc44r" podUID="d65e47d2-4461-406a-9b71-bb3fac4262f0" containerName="main" probeResult="failure" output="Get \"https://10.132.0.93:8000/ready\": context deadline exceeded"
Oct 30 13:20:05.275435 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:05.262997    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w2-f76c09/distinct-2-w2-f76c09-kserve-78446dcc4c-7ljvf"]
Oct 30 13:20:05.324761 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:05.265500    4943 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-ab3176/simulator-w6-ab3176-kserve-6d9b64549f-dc8xz" podUID="da306792-4f80-4df4-aa44-82991ab856e4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.69:8000/ready\": context deadline exceeded"
Oct 30 13:20:05.487321 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:05.268079    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-f76c09/distinct-2-w2-f76c09-kserve-78446dcc4c-7ljvf"
Oct 30 13:20:05.543781 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:05.543681    4943 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-d0a18c/distinct-w3-d0a18c-kserve-86479f6bbf-5zcqq" podUID="63dace4c-7832-47dd-84ab-2c91483a5563" containerName="main" probeResult="failure" output="Get \"https://10.132.0.89:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:20:05.624055 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:05.571056    4943 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w2-f76c09\"/\"default-dockercfg-6fxwq\""
Oct 30 13:20:05.624055 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:05.571512    4943 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w2-f76c09\"/\"distinct-2-w2-f76c09-kserve-self-signed-certs\""
Oct 30 13:20:05.624055 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:05.571749    4943 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w2-f76c09\"/\"kube-root-ca.crt\""
Oct 30 13:20:05.624055 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:05.571982    4943 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w2-f76c09\"/\"openshift-service-ca.crt\""
Oct 30 13:20:05.624055 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:05.618987    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9bcea043-6bca-43ef-8aa8-f4a82cc98b1a-tmp-dir\") pod \"distinct-2-w2-f76c09-kserve-78446dcc4c-7ljvf\" (UID: \"9bcea043-6bca-43ef-8aa8-f4a82cc98b1a\") " pod="e2e-models-e2e-worker-w2-f76c09/distinct-2-w2-f76c09-kserve-78446dcc4c-7ljvf"
Oct 30 13:20:05.624055 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:05.619097    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9bcea043-6bca-43ef-8aa8-f4a82cc98b1a-dshm\") pod \"distinct-2-w2-f76c09-kserve-78446dcc4c-7ljvf\" (UID: \"9bcea043-6bca-43ef-8aa8-f4a82cc98b1a\") " pod="e2e-models-e2e-worker-w2-f76c09/distinct-2-w2-f76c09-kserve-78446dcc4c-7ljvf"
Oct 30 13:20:05.624055 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:05.619198    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9bcea043-6bca-43ef-8aa8-f4a82cc98b1a-model-cache\") pod \"distinct-2-w2-f76c09-kserve-78446dcc4c-7ljvf\" (UID: \"9bcea043-6bca-43ef-8aa8-f4a82cc98b1a\") " pod="e2e-models-e2e-worker-w2-f76c09/distinct-2-w2-f76c09-kserve-78446dcc4c-7ljvf"
Oct 30 13:20:05.624055 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:05.619291    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c8zbd\" (UniqueName: \"kubernetes.io/projected/9bcea043-6bca-43ef-8aa8-f4a82cc98b1a-kube-api-access-c8zbd\") pod \"distinct-2-w2-f76c09-kserve-78446dcc4c-7ljvf\" (UID: \"9bcea043-6bca-43ef-8aa8-f4a82cc98b1a\") " pod="e2e-models-e2e-worker-w2-f76c09/distinct-2-w2-f76c09-kserve-78446dcc4c-7ljvf"
Oct 30 13:20:05.624055 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:05.619366    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9bcea043-6bca-43ef-8aa8-f4a82cc98b1a-home\") pod \"distinct-2-w2-f76c09-kserve-78446dcc4c-7ljvf\" (UID: \"9bcea043-6bca-43ef-8aa8-f4a82cc98b1a\") " pod="e2e-models-e2e-worker-w2-f76c09/distinct-2-w2-f76c09-kserve-78446dcc4c-7ljvf"
Oct 30 13:20:05.624055 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:05.619442    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9bcea043-6bca-43ef-8aa8-f4a82cc98b1a-tls-certs\") pod \"distinct-2-w2-f76c09-kserve-78446dcc4c-7ljvf\" (UID: \"9bcea043-6bca-43ef-8aa8-f4a82cc98b1a\") " pod="e2e-models-e2e-worker-w2-f76c09/distinct-2-w2-f76c09-kserve-78446dcc4c-7ljvf"
Oct 30 13:20:05.754283 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:05.753621    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9bcea043-6bca-43ef-8aa8-f4a82cc98b1a-model-cache\") pod \"distinct-2-w2-f76c09-kserve-78446dcc4c-7ljvf\" (UID: \"9bcea043-6bca-43ef-8aa8-f4a82cc98b1a\") " pod="e2e-models-e2e-worker-w2-f76c09/distinct-2-w2-f76c09-kserve-78446dcc4c-7ljvf"
Oct 30 13:20:05.754283 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:05.753726    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c8zbd\" (UniqueName: \"kubernetes.io/projected/9bcea043-6bca-43ef-8aa8-f4a82cc98b1a-kube-api-access-c8zbd\") pod \"distinct-2-w2-f76c09-kserve-78446dcc4c-7ljvf\" (UID: \"9bcea043-6bca-43ef-8aa8-f4a82cc98b1a\") " pod="e2e-models-e2e-worker-w2-f76c09/distinct-2-w2-f76c09-kserve-78446dcc4c-7ljvf"
Oct 30 13:20:05.754283 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:05.753800    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9bcea043-6bca-43ef-8aa8-f4a82cc98b1a-home\") pod \"distinct-2-w2-f76c09-kserve-78446dcc4c-7ljvf\" (UID: \"9bcea043-6bca-43ef-8aa8-f4a82cc98b1a\") " pod="e2e-models-e2e-worker-w2-f76c09/distinct-2-w2-f76c09-kserve-78446dcc4c-7ljvf"
Oct 30 13:20:05.754283 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:05.753877    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9bcea043-6bca-43ef-8aa8-f4a82cc98b1a-tls-certs\") pod \"distinct-2-w2-f76c09-kserve-78446dcc4c-7ljvf\" (UID: \"9bcea043-6bca-43ef-8aa8-f4a82cc98b1a\") " pod="e2e-models-e2e-worker-w2-f76c09/distinct-2-w2-f76c09-kserve-78446dcc4c-7ljvf"
Oct 30 13:20:05.872174 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:05.713274    4943 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w6-ab3176/simulator-w6-ab3176-kserve-6d9b64549f-dc8xz" podUID="da306792-4f80-4df4-aa44-82991ab856e4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.69:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:20:05.963326 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:05.868739    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9bcea043-6bca-43ef-8aa8-f4a82cc98b1a-tmp-dir\") pod \"distinct-2-w2-f76c09-kserve-78446dcc4c-7ljvf\" (UID: \"9bcea043-6bca-43ef-8aa8-f4a82cc98b1a\") " pod="e2e-models-e2e-worker-w2-f76c09/distinct-2-w2-f76c09-kserve-78446dcc4c-7ljvf"
Oct 30 13:20:05.963326 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:05.763566    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9bcea043-6bca-43ef-8aa8-f4a82cc98b1a-home\") pod \"distinct-2-w2-f76c09-kserve-78446dcc4c-7ljvf\" (UID: \"9bcea043-6bca-43ef-8aa8-f4a82cc98b1a\") " pod="e2e-models-e2e-worker-w2-f76c09/distinct-2-w2-f76c09-kserve-78446dcc4c-7ljvf"
Oct 30 13:20:05.963326 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:05.801994    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w3-d0a18c-b6l49" event={"ID":"08f92795-8d3e-465d-8571-1b4f5a0133b3","Type":"ContainerStarted","Data":"a9fe59189286bf29381807b045ab5340e2b8de4d4308bb7715506e303da604db"}
Oct 30 13:20:05.963326 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:05.876696    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9bcea043-6bca-43ef-8aa8-f4a82cc98b1a-tmp-dir\") pod \"distinct-2-w2-f76c09-kserve-78446dcc4c-7ljvf\" (UID: \"9bcea043-6bca-43ef-8aa8-f4a82cc98b1a\") " pod="e2e-models-e2e-worker-w2-f76c09/distinct-2-w2-f76c09-kserve-78446dcc4c-7ljvf"
Oct 30 13:20:05.963326 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:05.876789    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9bcea043-6bca-43ef-8aa8-f4a82cc98b1a-dshm\") pod \"distinct-2-w2-f76c09-kserve-78446dcc4c-7ljvf\" (UID: \"9bcea043-6bca-43ef-8aa8-f4a82cc98b1a\") " pod="e2e-models-e2e-worker-w2-f76c09/distinct-2-w2-f76c09-kserve-78446dcc4c-7ljvf"
Oct 30 13:20:05.963326 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:05.763250    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9bcea043-6bca-43ef-8aa8-f4a82cc98b1a-model-cache\") pod \"distinct-2-w2-f76c09-kserve-78446dcc4c-7ljvf\" (UID: \"9bcea043-6bca-43ef-8aa8-f4a82cc98b1a\") " pod="e2e-models-e2e-worker-w2-f76c09/distinct-2-w2-f76c09-kserve-78446dcc4c-7ljvf"
Oct 30 13:20:05.963326 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:05.897913    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w3-d0a18c-b6l49"]
Oct 30 13:20:05.963326 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:05.917233    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c8zbd\" (UniqueName: \"kubernetes.io/projected/9bcea043-6bca-43ef-8aa8-f4a82cc98b1a-kube-api-access-c8zbd\") pod \"distinct-2-w2-f76c09-kserve-78446dcc4c-7ljvf\" (UID: \"9bcea043-6bca-43ef-8aa8-f4a82cc98b1a\") " pod="e2e-models-e2e-worker-w2-f76c09/distinct-2-w2-f76c09-kserve-78446dcc4c-7ljvf"
Oct 30 13:20:06.104305 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:06.072553    4943 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-d0a18c/distinct-2-w3-d0a18c-kserve-85f8ff4846-rc44r" podUID="d65e47d2-4461-406a-9b71-bb3fac4262f0" containerName="main" probeResult="failure" output="Get \"https://10.132.0.93:8000/ready\": context deadline exceeded"
Oct 30 13:20:06.162658 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:06.162626    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9bcea043-6bca-43ef-8aa8-f4a82cc98b1a-dshm\") pod \"distinct-2-w2-f76c09-kserve-78446dcc4c-7ljvf\" (UID: \"9bcea043-6bca-43ef-8aa8-f4a82cc98b1a\") " pod="e2e-models-e2e-worker-w2-f76c09/distinct-2-w2-f76c09-kserve-78446dcc4c-7ljvf"
Oct 30 13:20:06.318963 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:06.318916    4943 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-ab3176/simulator-w6-ab3176-kserve-6d9b64549f-dc8xz" podUID="da306792-4f80-4df4-aa44-82991ab856e4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.69:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:20:06.567989 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:06.566754    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9bcea043-6bca-43ef-8aa8-f4a82cc98b1a-tls-certs\") pod \"distinct-2-w2-f76c09-kserve-78446dcc4c-7ljvf\" (UID: \"9bcea043-6bca-43ef-8aa8-f4a82cc98b1a\") " pod="e2e-models-e2e-worker-w2-f76c09/distinct-2-w2-f76c09-kserve-78446dcc4c-7ljvf"
Oct 30 13:20:06.668063 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:06.668012    4943 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 13:20:06.668302 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:06.668279    4943 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 13:20:06.668474 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:06.668454    4943 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 13:20:06.730010 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:06.727084    4943 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w3-d0a18c/distinct-2-w3-d0a18c-kserve-85f8ff4846-rc44r"]
Oct 30 13:20:06.730010 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:06.727129    4943 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w3-d0a18c/distinct-w3-d0a18c-kserve-86479f6bbf-5zcqq"]
Oct 30 13:20:06.730010 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:06.727187    4943 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-d0a18c/distinct-2-w3-d0a18c-kserve-85f8ff4846-rc44r"
Oct 30 13:20:06.730010 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:06.727217    4943 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-d0a18c/distinct-w3-d0a18c-kserve-86479f6bbf-5zcqq"
Oct 30 13:20:06.730010 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:06.727232    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-ljkqn"]
Oct 30 13:20:06.807891 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:06.787144    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-f76c09/distinct-2-w2-f76c09-kserve-78446dcc4c-7ljvf"
Oct 30 13:20:06.807891 ip-10-0-142-84 kubenswrapper[4943]: E1030 13:20:06.793767    4943 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-knqz6" containerName="extractor"
Oct 30 13:20:06.995623 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:06.995578    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-ljkqn" event={"ID":"f0ab36b0-4ba8-41da-8fc2-6a70d1082599","Type":"ContainerStarted","Data":"1a0d458f0843f21a6f358dd72f69723eda6aad9fc1cdd4c569327c01bc4921be"}
Oct 30 13:20:07.051320 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:07.034809    4943 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-ab3176/simulator-w6-ab3176-kserve-6d9b64549f-dc8xz"
Oct 30 13:20:07.148679 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:07.145575    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-pph7x"]
Oct 30 13:20:07.170772 ip-10-0-142-84 kubenswrapper[4943]: W1030 13:20:07.164423    4943 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbd83c13a_8f22_4ada_a894_90cd49e87782.slice/crio-5add7b0434636953d0383351f4a97b926da5985db410201394c9d68df1ba57fc WatchSource:0}: Error finding container 5add7b0434636953d0383351f4a97b926da5985db410201394c9d68df1ba57fc: Status 404 returned error can't find the container with id 5add7b0434636953d0383351f4a97b926da5985db410201394c9d68df1ba57fc
Oct 30 13:20:07.178234 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:07.177795    4943 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 13:20:07.178234 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:07.177888    4943 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 13:20:07.178234 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:07.177967    4943 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 13:20:07.502804 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:07.502771    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w2-f76c09/distinct-2-w2-f76c09-kserve-78446dcc4c-7ljvf"]
Oct 30 13:20:07.503533 ip-10-0-142-84 kubenswrapper[4943]: W1030 13:20:07.503503    4943 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9bcea043_6bca_43ef_8aa8_f4a82cc98b1a.slice/crio-014e15ea14cf1178b9405848957a09008ebbf444de422fd1e15b826e4f74c6fd WatchSource:0}: Error finding container 014e15ea14cf1178b9405848957a09008ebbf444de422fd1e15b826e4f74c6fd: Status 404 returned error can't find the container with id 014e15ea14cf1178b9405848957a09008ebbf444de422fd1e15b826e4f74c6fd
Oct 30 13:20:07.653165 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:07.653128    4943 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-vqzsp"]
Oct 30 13:20:07.657787 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:07.657763    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-vqzsp"
Oct 30 13:20:07.696489 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:07.696450    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-vqzsp"]
Oct 30 13:20:07.804204 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:07.804088    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6b9fc86d-7710-41b3-8d41-35cb717eb50e-workload-socket\") pod \"e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-vqzsp\" (UID: \"6b9fc86d-7710-41b3-8d41-35cb717eb50e\") " pod="openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-vqzsp"
Oct 30 13:20:07.804506 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:07.804475    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6b9fc86d-7710-41b3-8d41-35cb717eb50e-istio-data\") pod \"e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-vqzsp\" (UID: \"6b9fc86d-7710-41b3-8d41-35cb717eb50e\") " pod="openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-vqzsp"
Oct 30 13:20:07.804634 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:07.804617    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6b9fc86d-7710-41b3-8d41-35cb717eb50e-istio-podinfo\") pod \"e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-vqzsp\" (UID: \"6b9fc86d-7710-41b3-8d41-35cb717eb50e\") " pod="openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-vqzsp"
Oct 30 13:20:07.804729 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:07.804717    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6b9fc86d-7710-41b3-8d41-35cb717eb50e-istio-token\") pod \"e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-vqzsp\" (UID: \"6b9fc86d-7710-41b3-8d41-35cb717eb50e\") " pod="openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-vqzsp"
Oct 30 13:20:07.804917 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:07.804903    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cz4jn\" (UniqueName: \"kubernetes.io/projected/6b9fc86d-7710-41b3-8d41-35cb717eb50e-kube-api-access-cz4jn\") pod \"e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-vqzsp\" (UID: \"6b9fc86d-7710-41b3-8d41-35cb717eb50e\") " pod="openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-vqzsp"
Oct 30 13:20:07.805051 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:07.805037    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6b9fc86d-7710-41b3-8d41-35cb717eb50e-credential-socket\") pod \"e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-vqzsp\" (UID: \"6b9fc86d-7710-41b3-8d41-35cb717eb50e\") " pod="openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-vqzsp"
Oct 30 13:20:07.805305 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:07.805287    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6b9fc86d-7710-41b3-8d41-35cb717eb50e-istio-envoy\") pod \"e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-vqzsp\" (UID: \"6b9fc86d-7710-41b3-8d41-35cb717eb50e\") " pod="openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-vqzsp"
Oct 30 13:20:07.805499 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:07.805485    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6b9fc86d-7710-41b3-8d41-35cb717eb50e-istiod-ca-cert\") pod \"e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-vqzsp\" (UID: \"6b9fc86d-7710-41b3-8d41-35cb717eb50e\") " pod="openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-vqzsp"
Oct 30 13:20:07.805602 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:07.805590    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6b9fc86d-7710-41b3-8d41-35cb717eb50e-workload-certs\") pod \"e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-vqzsp\" (UID: \"6b9fc86d-7710-41b3-8d41-35cb717eb50e\") " pod="openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-vqzsp"
Oct 30 13:20:07.908779 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:07.908472    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cz4jn\" (UniqueName: \"kubernetes.io/projected/6b9fc86d-7710-41b3-8d41-35cb717eb50e-kube-api-access-cz4jn\") pod \"e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-vqzsp\" (UID: \"6b9fc86d-7710-41b3-8d41-35cb717eb50e\") " pod="openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-vqzsp"
Oct 30 13:20:07.908779 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:07.908537    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6b9fc86d-7710-41b3-8d41-35cb717eb50e-credential-socket\") pod \"e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-vqzsp\" (UID: \"6b9fc86d-7710-41b3-8d41-35cb717eb50e\") " pod="openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-vqzsp"
Oct 30 13:20:07.908779 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:07.908625    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6b9fc86d-7710-41b3-8d41-35cb717eb50e-istio-envoy\") pod \"e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-vqzsp\" (UID: \"6b9fc86d-7710-41b3-8d41-35cb717eb50e\") " pod="openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-vqzsp"
Oct 30 13:20:07.908779 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:07.908730    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6b9fc86d-7710-41b3-8d41-35cb717eb50e-istiod-ca-cert\") pod \"e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-vqzsp\" (UID: \"6b9fc86d-7710-41b3-8d41-35cb717eb50e\") " pod="openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-vqzsp"
Oct 30 13:20:07.908779 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:07.908772    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6b9fc86d-7710-41b3-8d41-35cb717eb50e-workload-certs\") pod \"e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-vqzsp\" (UID: \"6b9fc86d-7710-41b3-8d41-35cb717eb50e\") " pod="openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-vqzsp"
Oct 30 13:20:07.909193 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:07.908841    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6b9fc86d-7710-41b3-8d41-35cb717eb50e-workload-socket\") pod \"e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-vqzsp\" (UID: \"6b9fc86d-7710-41b3-8d41-35cb717eb50e\") " pod="openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-vqzsp"
Oct 30 13:20:07.909193 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:07.908882    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6b9fc86d-7710-41b3-8d41-35cb717eb50e-istio-data\") pod \"e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-vqzsp\" (UID: \"6b9fc86d-7710-41b3-8d41-35cb717eb50e\") " pod="openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-vqzsp"
Oct 30 13:20:07.909193 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:07.908918    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6b9fc86d-7710-41b3-8d41-35cb717eb50e-istio-podinfo\") pod \"e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-vqzsp\" (UID: \"6b9fc86d-7710-41b3-8d41-35cb717eb50e\") " pod="openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-vqzsp"
Oct 30 13:20:07.909193 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:07.908944    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6b9fc86d-7710-41b3-8d41-35cb717eb50e-istio-token\") pod \"e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-vqzsp\" (UID: \"6b9fc86d-7710-41b3-8d41-35cb717eb50e\") " pod="openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-vqzsp"
Oct 30 13:20:07.909982 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:07.909833    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6b9fc86d-7710-41b3-8d41-35cb717eb50e-workload-certs\") pod \"e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-vqzsp\" (UID: \"6b9fc86d-7710-41b3-8d41-35cb717eb50e\") " pod="openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-vqzsp"
Oct 30 13:20:07.910883 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:07.910535    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6b9fc86d-7710-41b3-8d41-35cb717eb50e-credential-socket\") pod \"e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-vqzsp\" (UID: \"6b9fc86d-7710-41b3-8d41-35cb717eb50e\") " pod="openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-vqzsp"
Oct 30 13:20:07.910883 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:07.910846    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6b9fc86d-7710-41b3-8d41-35cb717eb50e-istio-data\") pod \"e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-vqzsp\" (UID: \"6b9fc86d-7710-41b3-8d41-35cb717eb50e\") " pod="openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-vqzsp"
Oct 30 13:20:07.911392 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:07.911348    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6b9fc86d-7710-41b3-8d41-35cb717eb50e-istiod-ca-cert\") pod \"e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-vqzsp\" (UID: \"6b9fc86d-7710-41b3-8d41-35cb717eb50e\") " pod="openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-vqzsp"
Oct 30 13:20:07.911616 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:07.911595    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6b9fc86d-7710-41b3-8d41-35cb717eb50e-workload-socket\") pod \"e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-vqzsp\" (UID: \"6b9fc86d-7710-41b3-8d41-35cb717eb50e\") " pod="openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-vqzsp"
Oct 30 13:20:07.914591 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:07.914545    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6b9fc86d-7710-41b3-8d41-35cb717eb50e-istio-envoy\") pod \"e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-vqzsp\" (UID: \"6b9fc86d-7710-41b3-8d41-35cb717eb50e\") " pod="openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-vqzsp"
Oct 30 13:20:07.914798 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:07.914777    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6b9fc86d-7710-41b3-8d41-35cb717eb50e-istio-podinfo\") pod \"e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-vqzsp\" (UID: \"6b9fc86d-7710-41b3-8d41-35cb717eb50e\") " pod="openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-vqzsp"
Oct 30 13:20:07.934022 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:07.933998    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cz4jn\" (UniqueName: \"kubernetes.io/projected/6b9fc86d-7710-41b3-8d41-35cb717eb50e-kube-api-access-cz4jn\") pod \"e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-vqzsp\" (UID: \"6b9fc86d-7710-41b3-8d41-35cb717eb50e\") " pod="openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-vqzsp"
Oct 30 13:20:07.934274 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:07.934239    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6b9fc86d-7710-41b3-8d41-35cb717eb50e-istio-token\") pod \"e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-vqzsp\" (UID: \"6b9fc86d-7710-41b3-8d41-35cb717eb50e\") " pod="openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-vqzsp"
Oct 30 13:20:07.984927 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:07.984885    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-vqzsp"
Oct 30 13:20:08.022077 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:08.022041    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-pph7x" event={"ID":"bd83c13a-8f22-4ada-a894-90cd49e87782","Type":"ContainerStarted","Data":"c3864650b233dda9248352e7bb7d5bccbe2291f3806423c81a128038986a9903"}
Oct 30 13:20:08.022292 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:08.022097    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-pph7x" event={"ID":"bd83c13a-8f22-4ada-a894-90cd49e87782","Type":"ContainerStarted","Data":"5add7b0434636953d0383351f4a97b926da5985db410201394c9d68df1ba57fc"}
Oct 30 13:20:08.024151 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:08.024120    4943 generic.go:356] "Generic (PLEG): container finished" podID="08f92795-8d3e-465d-8571-1b4f5a0133b3" containerID="bd34232f1d63e999d6cd1d3fb1395fddfe7a0610320ec2bb7799d7ee9092f793" exitCode=0
Oct 30 13:20:08.024276 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:08.024201    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w3-d0a18c-b6l49" event={"ID":"08f92795-8d3e-465d-8571-1b4f5a0133b3","Type":"ContainerDied","Data":"bd34232f1d63e999d6cd1d3fb1395fddfe7a0610320ec2bb7799d7ee9092f793"}
Oct 30 13:20:08.026075 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:08.026049    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-f76c09/distinct-2-w2-f76c09-kserve-78446dcc4c-7ljvf" event={"ID":"9bcea043-6bca-43ef-8aa8-f4a82cc98b1a","Type":"ContainerStarted","Data":"561eb3fc9f77813de62ee57c5cd76b786f814901f9693f9945837a4ed517f533"}
Oct 30 13:20:08.026162 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:08.026088    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-f76c09/distinct-2-w2-f76c09-kserve-78446dcc4c-7ljvf" event={"ID":"9bcea043-6bca-43ef-8aa8-f4a82cc98b1a","Type":"ContainerStarted","Data":"014e15ea14cf1178b9405848957a09008ebbf444de422fd1e15b826e4f74c6fd"}
Oct 30 13:20:08.028183 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:08.028159    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-ljkqn" event={"ID":"f0ab36b0-4ba8-41da-8fc2-6a70d1082599","Type":"ContainerStarted","Data":"3bb47869cf566fc54232cdffa43db4a7042cefe5994382c6cec1269971649d61"}
Oct 30 13:20:08.128801 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:08.127803    4943 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-2e9a65a0-42gwd"]
Oct 30 13:20:08.136322 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:08.136295    4943 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-2e9a65a0-42gwd"]
Oct 30 13:20:08.140580 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:08.140537    4943 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-pph7x" podStartSLOduration=7.140521575 podStartE2EDuration="7.140521575s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:20:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:20:08.07532741 +0000 UTC m=+1383.863331142" watchObservedRunningTime="2026-10-30 13:20:08.140521575 +0000 UTC m=+1383.928525306"
Oct 30 13:20:08.147468 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:08.147432    4943 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-ljkqn" podStartSLOduration=7.147419664 podStartE2EDuration="7.147419664s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:20:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:20:08.144079768 +0000 UTC m=+1383.932083513" watchObservedRunningTime="2026-10-30 13:20:08.147419664 +0000 UTC m=+1383.935423393"
Oct 30 13:20:08.152415 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:08.152208    4943 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-ljkqn"
Oct 30 13:20:08.152491 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:08.152449    4943 patch_prober.go:29] interesting pod/e2e-worker-w2-f76c09-openshift-default-65d5845b67-ljkqn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.118:15021/healthz/ready\": dial tcp 10.132.0.118:15021: connect: connection refused" start-of-body=
Oct 30 13:20:08.152541 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:08.152493    4943 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-ljkqn" podUID="f0ab36b0-4ba8-41da-8fc2-6a70d1082599" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.118:15021/healthz/ready\": dial tcp 10.132.0.118:15021: connect: connection refused"
Oct 30 13:20:08.230359 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:08.230119    4943 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w2-f76c09/distinct-2-w2-f76c09-kserve-78446dcc4c-7ljvf" podStartSLOduration=5.230098664 podStartE2EDuration="5.230098664s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:20:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:20:08.215281685 +0000 UTC m=+1384.003285428" watchObservedRunningTime="2026-10-30 13:20:08.230098664 +0000 UTC m=+1384.018102396"
Oct 30 13:20:08.518304 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:08.503921    4943 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 13:20:08.519006 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:08.518949    4943 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 13:20:08.519234 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:08.519210    4943 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 13:20:08.519906 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:08.515444    4943 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="82ef0ce2-e735-491c-8489-509d7aeb82d4" path="/var/lib/kubelet/pods/82ef0ce2-e735-491c-8489-509d7aeb82d4/volumes"
Oct 30 13:20:08.520739 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:08.520721    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-vqzsp"]
Oct 30 13:20:08.810381 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:08.809104    4943 patch_prober.go:29] interesting pod/e2e-worker-w2-f76c09-openshift-default-65d5845b67-pph7x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.119:15021/healthz/ready\": dial tcp 10.132.0.119:15021: connect: connection refused" start-of-body=
Oct 30 13:20:08.810381 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:08.809182    4943 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-pph7x" podUID="bd83c13a-8f22-4ada-a894-90cd49e87782" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.119:15021/healthz/ready\": dial tcp 10.132.0.119:15021: connect: connection refused"
Oct 30 13:20:08.810381 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:08.809544    4943 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-pph7x"
Oct 30 13:20:08.827432 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:08.827397    4943 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-f76c09/distinct-2-w2-f76c09-kserve-78446dcc4c-7ljvf"
Oct 30 13:20:09.039018 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:09.038987    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-vqzsp" event={"ID":"6b9fc86d-7710-41b3-8d41-35cb717eb50e","Type":"ContainerStarted","Data":"8db59a7efd7b42cde501a97b371d8429e86a1f57f77b62ed00959cb067a43748"}
Oct 30 13:20:09.039330 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:09.039253    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-vqzsp" event={"ID":"6b9fc86d-7710-41b3-8d41-35cb717eb50e","Type":"ContainerStarted","Data":"c89f60127479615776b090e7e57f75c75e2f788753371b4cb5c00b0071497f64"}
Oct 30 13:20:09.107682 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:09.091410    4943 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-vqzsp" podStartSLOduration=2.091387566 podStartE2EDuration="2.091387566s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:20:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:20:09.090596111 +0000 UTC m=+1384.878599866" watchObservedRunningTime="2026-10-30 13:20:09.091387566 +0000 UTC m=+1384.879391297"
Oct 30 13:20:09.152071 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:09.152040    4943 patch_prober.go:29] interesting pod/e2e-worker-w2-f76c09-openshift-default-65d5845b67-ljkqn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.118:15021/healthz/ready\": dial tcp 10.132.0.118:15021: connect: connection refused" start-of-body=
Oct 30 13:20:09.152395 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:09.152247    4943 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-ljkqn" podUID="f0ab36b0-4ba8-41da-8fc2-6a70d1082599" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.118:15021/healthz/ready\": dial tcp 10.132.0.118:15021: connect: connection refused"
Oct 30 13:20:09.434452 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:09.434054    4943 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w3-d0a18c-b6l49"
Oct 30 13:20:09.457174 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:09.457138    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/08f92795-8d3e-465d-8571-1b4f5a0133b3-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/08f92795-8d3e-465d-8571-1b4f5a0133b3-maas-api-openshift-ca\") pod \"08f92795-8d3e-465d-8571-1b4f5a0133b3\" (UID: \"08f92795-8d3e-465d-8571-1b4f5a0133b3\") "
Oct 30 13:20:09.457381 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:09.457201    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/08f92795-8d3e-465d-8571-1b4f5a0133b3-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/08f92795-8d3e-465d-8571-1b4f5a0133b3-maas-api-xks-ca\") pod \"08f92795-8d3e-465d-8571-1b4f5a0133b3\" (UID: \"08f92795-8d3e-465d-8571-1b4f5a0133b3\") "
Oct 30 13:20:09.457381 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:09.457362    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/08f92795-8d3e-465d-8571-1b4f5a0133b3-kube-api-access-j2dgm\" (UniqueName: \"kubernetes.io/projected/08f92795-8d3e-465d-8571-1b4f5a0133b3-kube-api-access-j2dgm\") pod \"08f92795-8d3e-465d-8571-1b4f5a0133b3\" (UID: \"08f92795-8d3e-465d-8571-1b4f5a0133b3\") "
Oct 30 13:20:09.457981 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:09.457948    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/08f92795-8d3e-465d-8571-1b4f5a0133b3-maas-api-openshift-ca" pod "08f92795-8d3e-465d-8571-1b4f5a0133b3" (UID: "08f92795-8d3e-465d-8571-1b4f5a0133b3"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 13:20:09.459925 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:09.459894    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/08f92795-8d3e-465d-8571-1b4f5a0133b3-kube-api-access-j2dgm" pod "08f92795-8d3e-465d-8571-1b4f5a0133b3" (UID: "08f92795-8d3e-465d-8571-1b4f5a0133b3"). InnerVolumeSpecName "kube-api-access-j2dgm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:20:09.460152 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:09.460112    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/08f92795-8d3e-465d-8571-1b4f5a0133b3-maas-api-xks-ca" pod "08f92795-8d3e-465d-8571-1b4f5a0133b3" (UID: "08f92795-8d3e-465d-8571-1b4f5a0133b3"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 13:20:09.560540 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:09.560486    4943 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/08f92795-8d3e-465d-8571-1b4f5a0133b3-maas-api-xks-ca\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:20:09.561109 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:09.561085    4943 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j2dgm\" (UniqueName: \"kubernetes.io/projected/08f92795-8d3e-465d-8571-1b4f5a0133b3-kube-api-access-j2dgm\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:20:09.573300 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:09.561241    4943 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/08f92795-8d3e-465d-8571-1b4f5a0133b3-maas-api-openshift-ca\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:20:09.841792 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:09.841695    4943 patch_prober.go:29] interesting pod/e2e-worker-w2-f76c09-openshift-default-65d5845b67-pph7x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.119:15021/healthz/ready\": dial tcp 10.132.0.119:15021: connect: connection refused" start-of-body=
Oct 30 13:20:09.842031 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:09.842002    4943 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-pph7x" podUID="bd83c13a-8f22-4ada-a894-90cd49e87782" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.119:15021/healthz/ready\": dial tcp 10.132.0.119:15021: connect: connection refused"
Oct 30 13:20:09.992615 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:09.992582    4943 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-vqzsp"
Oct 30 13:20:09.996410 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:09.996377    4943 patch_prober.go:29] interesting pod/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-vqzsp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.121:15021/healthz/ready\": dial tcp 10.132.0.121:15021: connect: connection refused" start-of-body=
Oct 30 13:20:09.996620 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:09.996591    4943 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-vqzsp" podUID="6b9fc86d-7710-41b3-8d41-35cb717eb50e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.121:15021/healthz/ready\": dial tcp 10.132.0.121:15021: connect: connection refused"
Oct 30 13:20:10.051500 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:10.051465    4943 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w3-d0a18c-b6l49"
Oct 30 13:20:10.051696 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:10.051509    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w3-d0a18c-b6l49" event={"ID":"08f92795-8d3e-465d-8571-1b4f5a0133b3","Type":"ContainerDied","Data":"a9fe59189286bf29381807b045ab5340e2b8de4d4308bb7715506e303da604db"}
Oct 30 13:20:10.051696 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:10.051571    4943 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a9fe59189286bf29381807b045ab5340e2b8de4d4308bb7715506e303da604db"
Oct 30 13:20:10.151566 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:10.151533    4943 patch_prober.go:29] interesting pod/e2e-worker-w2-f76c09-openshift-default-65d5845b67-ljkqn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.118:15021/healthz/ready\": dial tcp 10.132.0.118:15021: connect: connection refused" start-of-body=
Oct 30 13:20:10.151740 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:10.151593    4943 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-ljkqn" podUID="f0ab36b0-4ba8-41da-8fc2-6a70d1082599" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.118:15021/healthz/ready\": dial tcp 10.132.0.118:15021: connect: connection refused"
Oct 30 13:20:10.817465 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:10.817360    4943 patch_prober.go:29] interesting pod/e2e-worker-w2-f76c09-openshift-default-65d5845b67-pph7x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.119:15021/healthz/ready\": dial tcp 10.132.0.119:15021: connect: connection refused" start-of-body=
Oct 30 13:20:10.838285 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:10.830336    4943 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-pph7x" podUID="bd83c13a-8f22-4ada-a894-90cd49e87782" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.119:15021/healthz/ready\": dial tcp 10.132.0.119:15021: connect: connection refused"
Oct 30 13:20:10.952534 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:10.952507    4943 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w1-05942d-7d456fbd84-mvdr9"
Oct 30 13:20:10.987296 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:10.987117    4943 patch_prober.go:29] interesting pod/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-vqzsp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.121:15021/healthz/ready\": dial tcp 10.132.0.121:15021: connect: connection refused" start-of-body=
Oct 30 13:20:10.987296 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:10.987176    4943 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-vqzsp" podUID="6b9fc86d-7710-41b3-8d41-35cb717eb50e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.121:15021/healthz/ready\": dial tcp 10.132.0.121:15021: connect: connection refused"
Oct 30 13:20:11.054517 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:11.054464    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7607a99a-237c-4d0c-b5d0-5168ec6a6e3f-kube-api-access-ncqth\" (UniqueName: \"kubernetes.io/projected/7607a99a-237c-4d0c-b5d0-5168ec6a6e3f-kube-api-access-ncqth\") pod \"7607a99a-237c-4d0c-b5d0-5168ec6a6e3f\" (UID: \"7607a99a-237c-4d0c-b5d0-5168ec6a6e3f\") "
Oct 30 13:20:11.054686 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:11.054619    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/7607a99a-237c-4d0c-b5d0-5168ec6a6e3f-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7607a99a-237c-4d0c-b5d0-5168ec6a6e3f-plugins-config-volume\") pod \"7607a99a-237c-4d0c-b5d0-5168ec6a6e3f\" (UID: \"7607a99a-237c-4d0c-b5d0-5168ec6a6e3f\") "
Oct 30 13:20:11.055943 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:11.055909    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7607a99a-237c-4d0c-b5d0-5168ec6a6e3f-plugins-config-volume" pod "7607a99a-237c-4d0c-b5d0-5168ec6a6e3f" (UID: "7607a99a-237c-4d0c-b5d0-5168ec6a6e3f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 13:20:11.148634 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:11.148576    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7607a99a-237c-4d0c-b5d0-5168ec6a6e3f-kube-api-access-ncqth" pod "7607a99a-237c-4d0c-b5d0-5168ec6a6e3f" (UID: "7607a99a-237c-4d0c-b5d0-5168ec6a6e3f"). InnerVolumeSpecName "kube-api-access-ncqth". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:20:11.151724 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:11.151685    4943 patch_prober.go:29] interesting pod/e2e-worker-w2-f76c09-openshift-default-65d5845b67-ljkqn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.118:15021/healthz/ready\": dial tcp 10.132.0.118:15021: connect: connection refused" start-of-body=
Oct 30 13:20:11.151876 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:11.151764    4943 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-ljkqn" podUID="f0ab36b0-4ba8-41da-8fc2-6a70d1082599" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.118:15021/healthz/ready\": dial tcp 10.132.0.118:15021: connect: connection refused"
Oct 30 13:20:11.185284 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:11.163318    4943 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ncqth\" (UniqueName: \"kubernetes.io/projected/7607a99a-237c-4d0c-b5d0-5168ec6a6e3f-kube-api-access-ncqth\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:20:11.194401 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:11.194357    4943 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7607a99a-237c-4d0c-b5d0-5168ec6a6e3f-plugins-config-volume\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:20:11.197309 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:11.196969    4943 generic.go:356] "Generic (PLEG): container finished" podID="7607a99a-237c-4d0c-b5d0-5168ec6a6e3f" containerID="811b2cfece4fbf41e6834879aa348dada91ef3cafecf7876a28a4cf2b00d8d30" exitCode=0
Oct 30 13:20:11.223298 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:11.197607    4943 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w1-05942d-7d456fbd84-mvdr9"
Oct 30 13:20:11.245335 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:11.223925    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w1-05942d-7d456fbd84-mvdr9" event={"ID":"7607a99a-237c-4d0c-b5d0-5168ec6a6e3f","Type":"ContainerDied","Data":"811b2cfece4fbf41e6834879aa348dada91ef3cafecf7876a28a4cf2b00d8d30"}
Oct 30 13:20:11.245335 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:11.224040    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w1-05942d-7d456fbd84-mvdr9" event={"ID":"7607a99a-237c-4d0c-b5d0-5168ec6a6e3f","Type":"ContainerDied","Data":"cc046983286ad8903c5751524e4c09f74eda7e4ae7f7f7022b105aa95e74f83a"}
Oct 30 13:20:11.245335 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:11.224095    4943 scope.go:122] "RemoveContainer" containerID="811b2cfece4fbf41e6834879aa348dada91ef3cafecf7876a28a4cf2b00d8d30"
Oct 30 13:20:11.344495 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:11.344437    4943 scope.go:122] "RemoveContainer" containerID="811b2cfece4fbf41e6834879aa348dada91ef3cafecf7876a28a4cf2b00d8d30"
Oct 30 13:20:11.347439 ip-10-0-142-84 kubenswrapper[4943]: E1030 13:20:11.347406    4943 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"811b2cfece4fbf41e6834879aa348dada91ef3cafecf7876a28a4cf2b00d8d30\": container with ID starting with 811b2cfece4fbf41e6834879aa348dada91ef3cafecf7876a28a4cf2b00d8d30 not found: ID does not exist" containerID="811b2cfece4fbf41e6834879aa348dada91ef3cafecf7876a28a4cf2b00d8d30"
Oct 30 13:20:11.347554 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:11.347445    4943 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"811b2cfece4fbf41e6834879aa348dada91ef3cafecf7876a28a4cf2b00d8d30"} err="failed to get container status \"811b2cfece4fbf41e6834879aa348dada91ef3cafecf7876a28a4cf2b00d8d30\": rpc error: code = NotFound desc = could not find container \"811b2cfece4fbf41e6834879aa348dada91ef3cafecf7876a28a4cf2b00d8d30\": container with ID starting with 811b2cfece4fbf41e6834879aa348dada91ef3cafecf7876a28a4cf2b00d8d30 not found: ID does not exist"
Oct 30 13:20:11.349180 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:11.349159    4943 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w1-05942d-7d456fbd84-mvdr9"]
Oct 30 13:20:11.354062 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:11.354036    4943 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w1-05942d-7d456fbd84-mvdr9"]
Oct 30 13:20:11.652448 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:11.652368    4943 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-fbc73eb9-j9rdq"]
Oct 30 13:20:11.653021 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:11.653004    4943 cpu_manager.go:414] "containerMap: removing container" podUID="7607a99a-237c-4d0c-b5d0-5168ec6a6e3f" containerName="payload-processing"
Oct 30 13:20:11.653070 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:11.653022    4943 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7607a99a-237c-4d0c-b5d0-5168ec6a6e3f" containerName="payload-processing"
Oct 30 13:20:11.653070 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:11.653055    4943 cpu_manager.go:414] "containerMap: removing container" podUID="08f92795-8d3e-465d-8571-1b4f5a0133b3" containerName="revoke-keys"
Oct 30 13:20:11.653070 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:11.653060    4943 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="08f92795-8d3e-465d-8571-1b4f5a0133b3" containerName="revoke-keys"
Oct 30 13:20:11.653180 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:11.653144    4943 memory_manager.go:365] "RemoveStaleState removing state" podUID="7607a99a-237c-4d0c-b5d0-5168ec6a6e3f" containerName="payload-processing"
Oct 30 13:20:11.653180 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:11.653158    4943 memory_manager.go:365] "RemoveStaleState removing state" podUID="08f92795-8d3e-465d-8571-1b4f5a0133b3" containerName="revoke-keys"
Oct 30 13:20:11.658205 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:11.658184    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-fbc73eb9-j9rdq"
Oct 30 13:20:11.689509 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:11.688325    4943 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Oct 30 13:20:11.689509 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:11.688440    4943 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Oct 30 13:20:11.689509 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:11.688494    4943 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 13:20:11.689509 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:11.688570    4943 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-p9z57\""
Oct 30 13:20:11.720455 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:11.720417    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4cl5d\" (UniqueName: \"kubernetes.io/projected/ab205844-0e5c-4c60-846c-6f6627a10692-kube-api-access-4cl5d\") pod \"maas-api-revoke-sub-fbc73eb9-j9rdq\" (UID: \"ab205844-0e5c-4c60-846c-6f6627a10692\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-fbc73eb9-j9rdq"
Oct 30 13:20:11.720700 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:11.720676    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/ab205844-0e5c-4c60-846c-6f6627a10692-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-fbc73eb9-j9rdq\" (UID: \"ab205844-0e5c-4c60-846c-6f6627a10692\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-fbc73eb9-j9rdq"
Oct 30 13:20:11.736454 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:11.720840    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/ab205844-0e5c-4c60-846c-6f6627a10692-maas-api-xks-ca\") pod \"maas-api-revoke-sub-fbc73eb9-j9rdq\" (UID: \"ab205844-0e5c-4c60-846c-6f6627a10692\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-fbc73eb9-j9rdq"
Oct 30 13:20:11.754385 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:11.754349    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-fbc73eb9-j9rdq"]
Oct 30 13:20:11.818756 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:11.806448    4943 patch_prober.go:29] interesting pod/e2e-worker-w2-f76c09-openshift-default-65d5845b67-pph7x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.119:15021/healthz/ready\": dial tcp 10.132.0.119:15021: connect: connection refused" start-of-body=
Oct 30 13:20:11.818756 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:11.806519    4943 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-pph7x" podUID="bd83c13a-8f22-4ada-a894-90cd49e87782" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.119:15021/healthz/ready\": dial tcp 10.132.0.119:15021: connect: connection refused"
Oct 30 13:20:11.841893 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:11.841845    4943 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w3-d0a18c-778df4584c-4f9hx"]
Oct 30 13:20:11.845397 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:11.845366    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4cl5d\" (UniqueName: \"kubernetes.io/projected/ab205844-0e5c-4c60-846c-6f6627a10692-kube-api-access-4cl5d\") pod \"maas-api-revoke-sub-fbc73eb9-j9rdq\" (UID: \"ab205844-0e5c-4c60-846c-6f6627a10692\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-fbc73eb9-j9rdq"
Oct 30 13:20:11.845935 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:11.845910    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/ab205844-0e5c-4c60-846c-6f6627a10692-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-fbc73eb9-j9rdq\" (UID: \"ab205844-0e5c-4c60-846c-6f6627a10692\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-fbc73eb9-j9rdq"
Oct 30 13:20:11.847195 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:11.847173    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/ab205844-0e5c-4c60-846c-6f6627a10692-maas-api-xks-ca\") pod \"maas-api-revoke-sub-fbc73eb9-j9rdq\" (UID: \"ab205844-0e5c-4c60-846c-6f6627a10692\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-fbc73eb9-j9rdq"
Oct 30 13:20:11.847840 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:11.847092    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/ab205844-0e5c-4c60-846c-6f6627a10692-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-fbc73eb9-j9rdq\" (UID: \"ab205844-0e5c-4c60-846c-6f6627a10692\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-fbc73eb9-j9rdq"
Oct 30 13:20:11.852103 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:11.851953    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/ab205844-0e5c-4c60-846c-6f6627a10692-maas-api-xks-ca\") pod \"maas-api-revoke-sub-fbc73eb9-j9rdq\" (UID: \"ab205844-0e5c-4c60-846c-6f6627a10692\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-fbc73eb9-j9rdq"
Oct 30 13:20:11.902190 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:11.902155    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4cl5d\" (UniqueName: \"kubernetes.io/projected/ab205844-0e5c-4c60-846c-6f6627a10692-kube-api-access-4cl5d\") pod \"maas-api-revoke-sub-fbc73eb9-j9rdq\" (UID: \"ab205844-0e5c-4c60-846c-6f6627a10692\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-fbc73eb9-j9rdq"
Oct 30 13:20:11.994424 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:11.994381    4943 patch_prober.go:29] interesting pod/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-vqzsp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.121:15021/healthz/ready\": dial tcp 10.132.0.121:15021: connect: connection refused" start-of-body=
Oct 30 13:20:12.005127 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:12.005088    4943 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-vqzsp" podUID="6b9fc86d-7710-41b3-8d41-35cb717eb50e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.121:15021/healthz/ready\": dial tcp 10.132.0.121:15021: connect: connection refused"
Oct 30 13:20:12.030495 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:12.030463    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-fbc73eb9-j9rdq"
Oct 30 13:20:12.171494 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:12.171449    4943 patch_prober.go:29] interesting pod/e2e-worker-w2-f76c09-openshift-default-65d5845b67-ljkqn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.118:15021/healthz/ready\": dial tcp 10.132.0.118:15021: connect: connection refused" start-of-body=
Oct 30 13:20:12.182285 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:12.171745    4943 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-ljkqn" podUID="f0ab36b0-4ba8-41da-8fc2-6a70d1082599" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.118:15021/healthz/ready\": dial tcp 10.132.0.118:15021: connect: connection refused"
Oct 30 13:20:12.452303 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:12.452249    4943 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7607a99a-237c-4d0c-b5d0-5168ec6a6e3f" path="/var/lib/kubelet/pods/7607a99a-237c-4d0c-b5d0-5168ec6a6e3f/volumes"
Oct 30 13:20:12.961542 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:12.935430    4943 patch_prober.go:29] interesting pod/e2e-worker-w2-f76c09-openshift-default-65d5845b67-pph7x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.119:15021/healthz/ready\": dial tcp 10.132.0.119:15021: connect: connection refused" start-of-body=
Oct 30 13:20:12.961542 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:12.935495    4943 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-pph7x" podUID="bd83c13a-8f22-4ada-a894-90cd49e87782" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.119:15021/healthz/ready\": dial tcp 10.132.0.119:15021: connect: connection refused"
Oct 30 13:20:12.990827 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:12.990787    4943 patch_prober.go:29] interesting pod/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-vqzsp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.121:15021/healthz/ready\": dial tcp 10.132.0.121:15021: connect: connection refused" start-of-body=
Oct 30 13:20:13.005441 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:13.005394    4943 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-vqzsp" podUID="6b9fc86d-7710-41b3-8d41-35cb717eb50e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.121:15021/healthz/ready\": dial tcp 10.132.0.121:15021: connect: connection refused"
Oct 30 13:20:13.152425 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:13.152379    4943 patch_prober.go:29] interesting pod/e2e-worker-w2-f76c09-openshift-default-65d5845b67-ljkqn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.118:15021/healthz/ready\": dial tcp 10.132.0.118:15021: connect: connection refused" start-of-body=
Oct 30 13:20:13.152624 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:13.152469    4943 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-ljkqn" podUID="f0ab36b0-4ba8-41da-8fc2-6a70d1082599" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.118:15021/healthz/ready\": dial tcp 10.132.0.118:15021: connect: connection refused"
Oct 30 13:20:13.419295 ip-10-0-142-84 kubenswrapper[4943]: W1030 13:20:13.419184    4943 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podab205844_0e5c_4c60_846c_6f6627a10692.slice/crio-d08066c9bd8819bc9480355591295f884e57acdb460ee087fb9ea538c7fab08c WatchSource:0}: Error finding container d08066c9bd8819bc9480355591295f884e57acdb460ee087fb9ea538c7fab08c: Status 404 returned error can't find the container with id d08066c9bd8819bc9480355591295f884e57acdb460ee087fb9ea538c7fab08c
Oct 30 13:20:13.451284 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:13.447738    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-fbc73eb9-j9rdq"]
Oct 30 13:20:13.619340 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:13.618836    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-fbc73eb9-j9rdq" event={"ID":"ab205844-0e5c-4c60-846c-6f6627a10692","Type":"ContainerStarted","Data":"d08066c9bd8819bc9480355591295f884e57acdb460ee087fb9ea538c7fab08c"}
Oct 30 13:20:13.811736 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:13.811697    4943 patch_prober.go:29] interesting pod/e2e-worker-w2-f76c09-openshift-default-65d5845b67-pph7x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.119:15021/healthz/ready\": dial tcp 10.132.0.119:15021: connect: connection refused" start-of-body=
Oct 30 13:20:13.812069 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:13.812041    4943 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-pph7x" podUID="bd83c13a-8f22-4ada-a894-90cd49e87782" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.119:15021/healthz/ready\": dial tcp 10.132.0.119:15021: connect: connection refused"
Oct 30 13:20:13.990357 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:13.990312    4943 patch_prober.go:29] interesting pod/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-vqzsp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.121:15021/healthz/ready\": dial tcp 10.132.0.121:15021: connect: connection refused" start-of-body=
Oct 30 13:20:13.991094 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:13.991055    4943 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-vqzsp" podUID="6b9fc86d-7710-41b3-8d41-35cb717eb50e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.121:15021/healthz/ready\": dial tcp 10.132.0.121:15021: connect: connection refused"
Oct 30 13:20:14.151115 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:14.151080    4943 patch_prober.go:29] interesting pod/e2e-worker-w2-f76c09-openshift-default-65d5845b67-ljkqn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.118:15021/healthz/ready\": dial tcp 10.132.0.118:15021: connect: connection refused" start-of-body=
Oct 30 13:20:14.151567 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:14.151136    4943 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-ljkqn" podUID="f0ab36b0-4ba8-41da-8fc2-6a70d1082599" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.118:15021/healthz/ready\": dial tcp 10.132.0.118:15021: connect: connection refused"
Oct 30 13:20:14.722751 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:14.722715    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-fbc73eb9-j9rdq" event={"ID":"ab205844-0e5c-4c60-846c-6f6627a10692","Type":"ContainerStarted","Data":"93d2262feccf8b3f24095af2d9bb55ffbe292fc609c1ca7fb8d3310a77b08199"}
Oct 30 13:20:14.797285 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:14.791439    4943 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-fbc73eb9-j9rdq" podStartSLOduration=3.791417878 podStartE2EDuration="3.791417878s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:20:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:20:14.783014312 +0000 UTC m=+1390.571018043" watchObservedRunningTime="2026-10-30 13:20:14.791417878 +0000 UTC m=+1390.579421611"
Oct 30 13:20:14.814286 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:14.805387    4943 patch_prober.go:29] interesting pod/e2e-worker-w2-f76c09-openshift-default-65d5845b67-pph7x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.119:15021/healthz/ready\": dial tcp 10.132.0.119:15021: connect: connection refused" start-of-body=
Oct 30 13:20:14.814286 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:14.805448    4943 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-pph7x" podUID="bd83c13a-8f22-4ada-a894-90cd49e87782" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.119:15021/healthz/ready\": dial tcp 10.132.0.119:15021: connect: connection refused"
Oct 30 13:20:14.991335 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:14.989762    4943 patch_prober.go:29] interesting pod/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-vqzsp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.121:15021/healthz/ready\": dial tcp 10.132.0.121:15021: connect: connection refused" start-of-body=
Oct 30 13:20:14.991335 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:14.989838    4943 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-vqzsp" podUID="6b9fc86d-7710-41b3-8d41-35cb717eb50e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.121:15021/healthz/ready\": dial tcp 10.132.0.121:15021: connect: connection refused"
Oct 30 13:20:15.754038 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:15.754000    4943 generic.go:356] "Generic (PLEG): container finished" podID="ab205844-0e5c-4c60-846c-6f6627a10692" containerID="93d2262feccf8b3f24095af2d9bb55ffbe292fc609c1ca7fb8d3310a77b08199" exitCode=0
Oct 30 13:20:15.754354 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:15.754334    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-fbc73eb9-j9rdq" event={"ID":"ab205844-0e5c-4c60-846c-6f6627a10692","Type":"ContainerDied","Data":"93d2262feccf8b3f24095af2d9bb55ffbe292fc609c1ca7fb8d3310a77b08199"}
Oct 30 13:20:16.006346 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:15.994834    4943 patch_prober.go:29] interesting pod/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-vqzsp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.121:15021/healthz/ready\": dial tcp 10.132.0.121:15021: connect: connection refused" start-of-body=
Oct 30 13:20:16.006346 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:15.994895    4943 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-vqzsp" podUID="6b9fc86d-7710-41b3-8d41-35cb717eb50e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.121:15021/healthz/ready\": dial tcp 10.132.0.121:15021: connect: connection refused"
Oct 30 13:20:16.228644 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:16.228599    4943 patch_prober.go:29] interesting pod/e2e-worker-w2-f76c09-openshift-default-65d5845b67-ljkqn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.118:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 13:20:16.228952 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:16.228913    4943 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-ljkqn" podUID="f0ab36b0-4ba8-41da-8fc2-6a70d1082599" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.118:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:20:16.437635 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:16.437596    4943 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-ljkqn"
Oct 30 13:20:16.438159 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:16.438088    4943 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-ljkqn"
Oct 30 13:20:16.440634 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:16.440609    4943 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-ljkqn"
Oct 30 13:20:16.505282 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:16.488415    4943 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w4-88f6a4/distinct-2-w4-88f6a4-kserve-7b8f7bd7dc-5bt68"]
Oct 30 13:20:16.510058 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:16.509559    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-88f6a4/distinct-2-w4-88f6a4-kserve-7b8f7bd7dc-5bt68"
Oct 30 13:20:16.518409 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:16.518370    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-88f6a4/distinct-2-w4-88f6a4-kserve-7b8f7bd7dc-5bt68"]
Oct 30 13:20:16.534374 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:16.534322    4943 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-88f6a4\"/\"default-dockercfg-cdfd5\""
Oct 30 13:20:16.539477 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:16.539196    4943 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-88f6a4\"/\"openshift-service-ca.crt\""
Oct 30 13:20:16.539477 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:16.539356    4943 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-88f6a4\"/\"distinct-2-w4-88f6a4-kserve-self-signed-certs\""
Oct 30 13:20:16.539477 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:16.539445    4943 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-88f6a4\"/\"kube-root-ca.crt\""
Oct 30 13:20:16.581494 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:16.581147    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vgtqw\" (UniqueName: \"kubernetes.io/projected/fe434953-1a27-4c4f-9478-f8d0264c8f61-kube-api-access-vgtqw\") pod \"distinct-2-w4-88f6a4-kserve-7b8f7bd7dc-5bt68\" (UID: \"fe434953-1a27-4c4f-9478-f8d0264c8f61\") " pod="e2e-models-e2e-worker-w4-88f6a4/distinct-2-w4-88f6a4-kserve-7b8f7bd7dc-5bt68"
Oct 30 13:20:16.581793 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:16.581767    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fe434953-1a27-4c4f-9478-f8d0264c8f61-tls-certs\") pod \"distinct-2-w4-88f6a4-kserve-7b8f7bd7dc-5bt68\" (UID: \"fe434953-1a27-4c4f-9478-f8d0264c8f61\") " pod="e2e-models-e2e-worker-w4-88f6a4/distinct-2-w4-88f6a4-kserve-7b8f7bd7dc-5bt68"
Oct 30 13:20:16.581948 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:16.581930    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fe434953-1a27-4c4f-9478-f8d0264c8f61-dshm\") pod \"distinct-2-w4-88f6a4-kserve-7b8f7bd7dc-5bt68\" (UID: \"fe434953-1a27-4c4f-9478-f8d0264c8f61\") " pod="e2e-models-e2e-worker-w4-88f6a4/distinct-2-w4-88f6a4-kserve-7b8f7bd7dc-5bt68"
Oct 30 13:20:16.582067 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:16.582050    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fe434953-1a27-4c4f-9478-f8d0264c8f61-tmp-dir\") pod \"distinct-2-w4-88f6a4-kserve-7b8f7bd7dc-5bt68\" (UID: \"fe434953-1a27-4c4f-9478-f8d0264c8f61\") " pod="e2e-models-e2e-worker-w4-88f6a4/distinct-2-w4-88f6a4-kserve-7b8f7bd7dc-5bt68"
Oct 30 13:20:16.582423 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:16.582397    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fe434953-1a27-4c4f-9478-f8d0264c8f61-home\") pod \"distinct-2-w4-88f6a4-kserve-7b8f7bd7dc-5bt68\" (UID: \"fe434953-1a27-4c4f-9478-f8d0264c8f61\") " pod="e2e-models-e2e-worker-w4-88f6a4/distinct-2-w4-88f6a4-kserve-7b8f7bd7dc-5bt68"
Oct 30 13:20:16.582653 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:16.582633    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fe434953-1a27-4c4f-9478-f8d0264c8f61-model-cache\") pod \"distinct-2-w4-88f6a4-kserve-7b8f7bd7dc-5bt68\" (UID: \"fe434953-1a27-4c4f-9478-f8d0264c8f61\") " pod="e2e-models-e2e-worker-w4-88f6a4/distinct-2-w4-88f6a4-kserve-7b8f7bd7dc-5bt68"
Oct 30 13:20:16.784331 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:16.702134    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fe434953-1a27-4c4f-9478-f8d0264c8f61-model-cache\") pod \"distinct-2-w4-88f6a4-kserve-7b8f7bd7dc-5bt68\" (UID: \"fe434953-1a27-4c4f-9478-f8d0264c8f61\") " pod="e2e-models-e2e-worker-w4-88f6a4/distinct-2-w4-88f6a4-kserve-7b8f7bd7dc-5bt68"
Oct 30 13:20:16.784331 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:16.702284    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vgtqw\" (UniqueName: \"kubernetes.io/projected/fe434953-1a27-4c4f-9478-f8d0264c8f61-kube-api-access-vgtqw\") pod \"distinct-2-w4-88f6a4-kserve-7b8f7bd7dc-5bt68\" (UID: \"fe434953-1a27-4c4f-9478-f8d0264c8f61\") " pod="e2e-models-e2e-worker-w4-88f6a4/distinct-2-w4-88f6a4-kserve-7b8f7bd7dc-5bt68"
Oct 30 13:20:16.784331 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:16.702328    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fe434953-1a27-4c4f-9478-f8d0264c8f61-tls-certs\") pod \"distinct-2-w4-88f6a4-kserve-7b8f7bd7dc-5bt68\" (UID: \"fe434953-1a27-4c4f-9478-f8d0264c8f61\") " pod="e2e-models-e2e-worker-w4-88f6a4/distinct-2-w4-88f6a4-kserve-7b8f7bd7dc-5bt68"
Oct 30 13:20:16.784331 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:16.702388    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fe434953-1a27-4c4f-9478-f8d0264c8f61-dshm\") pod \"distinct-2-w4-88f6a4-kserve-7b8f7bd7dc-5bt68\" (UID: \"fe434953-1a27-4c4f-9478-f8d0264c8f61\") " pod="e2e-models-e2e-worker-w4-88f6a4/distinct-2-w4-88f6a4-kserve-7b8f7bd7dc-5bt68"
Oct 30 13:20:16.784331 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:16.702420    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fe434953-1a27-4c4f-9478-f8d0264c8f61-tmp-dir\") pod \"distinct-2-w4-88f6a4-kserve-7b8f7bd7dc-5bt68\" (UID: \"fe434953-1a27-4c4f-9478-f8d0264c8f61\") " pod="e2e-models-e2e-worker-w4-88f6a4/distinct-2-w4-88f6a4-kserve-7b8f7bd7dc-5bt68"
Oct 30 13:20:16.784331 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:16.702602    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fe434953-1a27-4c4f-9478-f8d0264c8f61-home\") pod \"distinct-2-w4-88f6a4-kserve-7b8f7bd7dc-5bt68\" (UID: \"fe434953-1a27-4c4f-9478-f8d0264c8f61\") " pod="e2e-models-e2e-worker-w4-88f6a4/distinct-2-w4-88f6a4-kserve-7b8f7bd7dc-5bt68"
Oct 30 13:20:16.784331 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:16.703125    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fe434953-1a27-4c4f-9478-f8d0264c8f61-home\") pod \"distinct-2-w4-88f6a4-kserve-7b8f7bd7dc-5bt68\" (UID: \"fe434953-1a27-4c4f-9478-f8d0264c8f61\") " pod="e2e-models-e2e-worker-w4-88f6a4/distinct-2-w4-88f6a4-kserve-7b8f7bd7dc-5bt68"
Oct 30 13:20:16.784331 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:16.703421    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fe434953-1a27-4c4f-9478-f8d0264c8f61-model-cache\") pod \"distinct-2-w4-88f6a4-kserve-7b8f7bd7dc-5bt68\" (UID: \"fe434953-1a27-4c4f-9478-f8d0264c8f61\") " pod="e2e-models-e2e-worker-w4-88f6a4/distinct-2-w4-88f6a4-kserve-7b8f7bd7dc-5bt68"
Oct 30 13:20:16.852813 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:16.807664    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fe434953-1a27-4c4f-9478-f8d0264c8f61-tls-certs\") pod \"distinct-2-w4-88f6a4-kserve-7b8f7bd7dc-5bt68\" (UID: \"fe434953-1a27-4c4f-9478-f8d0264c8f61\") " pod="e2e-models-e2e-worker-w4-88f6a4/distinct-2-w4-88f6a4-kserve-7b8f7bd7dc-5bt68"
Oct 30 13:20:16.852813 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:16.851022    4943 patch_prober.go:29] interesting pod/e2e-worker-w2-f76c09-openshift-default-65d5845b67-pph7x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.119:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 13:20:16.852813 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:16.851079    4943 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-pph7x" podUID="bd83c13a-8f22-4ada-a894-90cd49e87782" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.119:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:20:16.905303 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:16.868484    4943 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w2-f76c09/distinct-2-w2-f76c09-kserve-78446dcc4c-7ljvf"
Oct 30 13:20:16.905303 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:16.870313    4943 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-d0a18c-778df4584c-4f9hx" podUID="93220213-580a-465b-b060-3cea89bfd247" containerName="payload-pre-processing" containerID="cri-o://c02e281bfa1ce0adc4e192e59fa3f1afa783120d764a61804c6e5e1680cdd7bf" gracePeriod=25
Oct 30 13:20:16.905303 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:16.885616    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fe434953-1a27-4c4f-9478-f8d0264c8f61-tmp-dir\") pod \"distinct-2-w4-88f6a4-kserve-7b8f7bd7dc-5bt68\" (UID: \"fe434953-1a27-4c4f-9478-f8d0264c8f61\") " pod="e2e-models-e2e-worker-w4-88f6a4/distinct-2-w4-88f6a4-kserve-7b8f7bd7dc-5bt68"
Oct 30 13:20:16.968589 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:16.956718    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fe434953-1a27-4c4f-9478-f8d0264c8f61-dshm\") pod \"distinct-2-w4-88f6a4-kserve-7b8f7bd7dc-5bt68\" (UID: \"fe434953-1a27-4c4f-9478-f8d0264c8f61\") " pod="e2e-models-e2e-worker-w4-88f6a4/distinct-2-w4-88f6a4-kserve-7b8f7bd7dc-5bt68"
Oct 30 13:20:17.025475 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:17.023730    4943 patch_prober.go:29] interesting pod/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-vqzsp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.121:15021/healthz/ready\": dial tcp 10.132.0.121:15021: connect: connection refused" start-of-body=
Oct 30 13:20:17.025475 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:17.023801    4943 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-vqzsp" podUID="6b9fc86d-7710-41b3-8d41-35cb717eb50e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.121:15021/healthz/ready\": dial tcp 10.132.0.121:15021: connect: connection refused"
Oct 30 13:20:17.070253 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:17.070163    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vgtqw\" (UniqueName: \"kubernetes.io/projected/fe434953-1a27-4c4f-9478-f8d0264c8f61-kube-api-access-vgtqw\") pod \"distinct-2-w4-88f6a4-kserve-7b8f7bd7dc-5bt68\" (UID: \"fe434953-1a27-4c4f-9478-f8d0264c8f61\") " pod="e2e-models-e2e-worker-w4-88f6a4/distinct-2-w4-88f6a4-kserve-7b8f7bd7dc-5bt68"
Oct 30 13:20:17.150311 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:17.140974    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-88f6a4/distinct-2-w4-88f6a4-kserve-7b8f7bd7dc-5bt68"
Oct 30 13:20:17.166361 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:17.166318    4943 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w2-f76c09/distinct-2-w2-f76c09-kserve-78446dcc4c-7ljvf"
Oct 30 13:20:18.209073 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:18.209037    4943 patch_prober.go:29] interesting pod/e2e-worker-w2-f76c09-openshift-default-65d5845b67-pph7x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.119:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 13:20:18.237305 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:18.217683    4943 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-pph7x" podUID="bd83c13a-8f22-4ada-a894-90cd49e87782" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.119:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:20:18.237305 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:18.209616    4943 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-da16efe5-plpbz"]
Oct 30 13:20:18.314387 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:18.263164    4943 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-da16efe5-plpbz"]
Oct 30 13:20:18.314387 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:18.297861    4943 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-f76c09/distinct-2-w2-f76c09-kserve-78446dcc4c-7ljvf"
Oct 30 13:20:18.442291 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:18.429458    4943 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-d0a18c-778df4584c-4f9hx" podUID="93220213-580a-465b-b060-3cea89bfd247" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 13:20:18.503286 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:18.482749    4943 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-fbc73eb9-j9rdq"
Oct 30 13:20:18.625287 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:18.619128    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ab205844-0e5c-4c60-846c-6f6627a10692-kube-api-access-4cl5d\" (UniqueName: \"kubernetes.io/projected/ab205844-0e5c-4c60-846c-6f6627a10692-kube-api-access-4cl5d\") pod \"ab205844-0e5c-4c60-846c-6f6627a10692\" (UID: \"ab205844-0e5c-4c60-846c-6f6627a10692\") "
Oct 30 13:20:18.626019 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:18.625980    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ab205844-0e5c-4c60-846c-6f6627a10692-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/ab205844-0e5c-4c60-846c-6f6627a10692-maas-api-openshift-ca\") pod \"ab205844-0e5c-4c60-846c-6f6627a10692\" (UID: \"ab205844-0e5c-4c60-846c-6f6627a10692\") "
Oct 30 13:20:18.653589 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:18.646423    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ab205844-0e5c-4c60-846c-6f6627a10692-maas-api-openshift-ca" pod "ab205844-0e5c-4c60-846c-6f6627a10692" (UID: "ab205844-0e5c-4c60-846c-6f6627a10692"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 13:20:18.653943 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:18.653905    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/ab205844-0e5c-4c60-846c-6f6627a10692-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/ab205844-0e5c-4c60-846c-6f6627a10692-maas-api-xks-ca\") pod \"ab205844-0e5c-4c60-846c-6f6627a10692\" (UID: \"ab205844-0e5c-4c60-846c-6f6627a10692\") "
Oct 30 13:20:18.655611 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:18.655587    4943 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/ab205844-0e5c-4c60-846c-6f6627a10692-maas-api-openshift-ca\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:20:18.671286 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:18.671191    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ab205844-0e5c-4c60-846c-6f6627a10692-kube-api-access-4cl5d" pod "ab205844-0e5c-4c60-846c-6f6627a10692" (UID: "ab205844-0e5c-4c60-846c-6f6627a10692"). InnerVolumeSpecName "kube-api-access-4cl5d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:20:18.689319 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:18.677534    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ab205844-0e5c-4c60-846c-6f6627a10692-maas-api-xks-ca" pod "ab205844-0e5c-4c60-846c-6f6627a10692" (UID: "ab205844-0e5c-4c60-846c-6f6627a10692"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 13:20:18.792388 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:18.792348    4943 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4cl5d\" (UniqueName: \"kubernetes.io/projected/ab205844-0e5c-4c60-846c-6f6627a10692-kube-api-access-4cl5d\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:20:18.792629 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:18.792607    4943 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/ab205844-0e5c-4c60-846c-6f6627a10692-maas-api-xks-ca\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:20:18.815295 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:18.808700    4943 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5d850ac7-0bbd-466f-b3b2-c665eb07066f" path="/var/lib/kubelet/pods/5d850ac7-0bbd-466f-b3b2-c665eb07066f/volumes"
Oct 30 13:20:18.840473 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:18.840433    4943 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-vqzsp"
Oct 30 13:20:18.840885 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:18.840862    4943 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-pph7x"
Oct 30 13:20:18.843340 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:18.843312    4943 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-vqzsp"
Oct 30 13:20:18.847638 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:18.847621    4943 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-fbc73eb9-j9rdq"
Oct 30 13:20:18.852172 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:18.847857    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-fbc73eb9-j9rdq" event={"ID":"ab205844-0e5c-4c60-846c-6f6627a10692","Type":"ContainerDied","Data":"d08066c9bd8819bc9480355591295f884e57acdb460ee087fb9ea538c7fab08c"}
Oct 30 13:20:18.852576 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:18.852542    4943 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d08066c9bd8819bc9480355591295f884e57acdb460ee087fb9ea538c7fab08c"
Oct 30 13:20:18.852742 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:18.852717    4943 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-vqzsp"
Oct 30 13:20:18.884679 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:18.882968    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-88f6a4/distinct-2-w4-88f6a4-kserve-7b8f7bd7dc-5bt68"]
Oct 30 13:20:19.144707 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:19.144636    4943 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-pph7x"
Oct 30 13:20:19.855895 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:19.855860    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-88f6a4/distinct-2-w4-88f6a4-kserve-7b8f7bd7dc-5bt68" event={"ID":"fe434953-1a27-4c4f-9478-f8d0264c8f61","Type":"ContainerStarted","Data":"e4b545b668a00853f418cc56179cd7dc3276a4b4cb1f5368e68d2ee9f84ff492"}
Oct 30 13:20:19.856482 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:19.856448    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-88f6a4/distinct-2-w4-88f6a4-kserve-7b8f7bd7dc-5bt68" event={"ID":"fe434953-1a27-4c4f-9478-f8d0264c8f61","Type":"ContainerStarted","Data":"37bd20dc0155c4f8e7bf00c589a7888ef60f582897d64627c794dd0b34d75882"}
Oct 30 13:20:19.856960 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:19.856943    4943 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-pph7x"
Oct 30 13:20:19.906015 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:19.905973    4943 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w4-88f6a4/distinct-2-w4-88f6a4-kserve-7b8f7bd7dc-5bt68" podStartSLOduration=3.905961547 podStartE2EDuration="3.905961547s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:20:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:20:19.903976601 +0000 UTC m=+1395.691980331" watchObservedRunningTime="2026-10-30 13:20:19.905961547 +0000 UTC m=+1395.693965317"
Oct 30 13:20:20.143704 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:20.143531    4943 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-88f6a4/distinct-2-w4-88f6a4-kserve-7b8f7bd7dc-5bt68"
Oct 30 13:20:20.775456 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:20.775419    4943 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-glf25"]
Oct 30 13:20:20.775704 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:20.775676    4943 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-glf25" podUID="dbc82b1c-1c45-4263-b8ee-090fbecff086" containerName="istio-proxy" containerID="cri-o://08c663942fb9d9c7fa0105b756682ffc83a25223f84ca89191e85c5f7298e19c" gracePeriod=30
Oct 30 13:20:20.786690 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:20.786660    4943 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-58kq2"]
Oct 30 13:20:20.786860 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:20.786840    4943 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-58kq2" podUID="0b3caf04-1a9d-44ce-8d80-3335b28a36ab" containerName="istio-proxy" containerID="cri-o://82312698550d72f0e0cc6943d93c514b690a9e6ebc12f5d447d6cb49a5ba1d5b" gracePeriod=30
Oct 30 13:20:20.801592 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:20.801564    4943 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-f55w2"]
Oct 30 13:20:20.801762 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:20.801742    4943 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-f55w2" podUID="663524af-aa09-44fe-8186-654cb9668d27" containerName="istio-proxy" containerID="cri-o://a7aab891628438e6299302e8f696c682fa173c6902aa23e763ea7aa6424ad66a" gracePeriod=30
Oct 30 13:20:22.134826 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:22.134785    4943 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w3-d0a18c/distinct-w3-d0a18c-kserve-86479f6bbf-5zcqq" podUID="63dace4c-7832-47dd-84ab-2c91483a5563" containerName="main" containerID="cri-o://adeb6ab24fe6238092db30053af4cd8cc74a617b87301baf954b8382f346cd18" gracePeriod=45
Oct 30 13:20:22.137161 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:22.137129    4943 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w3-d0a18c/distinct-2-w3-d0a18c-kserve-85f8ff4846-rc44r" podUID="d65e47d2-4461-406a-9b71-bb3fac4262f0" containerName="main" containerID="cri-o://295b4280acd1ea46140fd906e618c2bee2291632e55178b166967be88fa38ce6" gracePeriod=45
Oct 30 13:20:22.446040 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:22.446012    4943 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-nbdn6"]
Oct 30 13:20:22.446226 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:22.446201    4943 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-nbdn6" podUID="7088cc53-0ec6-43be-9669-075ca0ad2fa1" containerName="istio-proxy" containerID="cri-o://6ada3baab87084aaeea903d02bf1eaba6e0b57952457b98756be7ec64be7cc28" gracePeriod=30
Oct 30 13:20:22.467211 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:22.467091    4943 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-c98s9"]
Oct 30 13:20:22.467505 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:22.467467    4943 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-c98s9" podUID="094f15a2-03fc-4322-9bcb-c190f5a6d8c4" containerName="istio-proxy" containerID="cri-o://d97dcbe0e2d9b367fb9764d646726b47dc3956da80a55b1a9441ee67cbd98ca9" gracePeriod=30
Oct 30 13:20:22.471509 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:22.471454    4943 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-d0a18c/distinct-w3-d0a18c-kserve-86479f6bbf-5zcqq"
Oct 30 13:20:22.474063 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:22.474042    4943 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-d0a18c/distinct-2-w3-d0a18c-kserve-85f8ff4846-rc44r"
Oct 30 13:20:22.474518 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:22.474495    4943 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-zmddf"]
Oct 30 13:20:22.474733 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:22.474709    4943 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-zmddf" podUID="78fecfa2-495c-4690-bd0e-08e0a4213ade" containerName="istio-proxy" containerID="cri-o://e119e449a536de7ed6161856fa5ecbae96bccab870c3312223cac6af51b4d68d" gracePeriod=30
Oct 30 13:20:22.544282 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:22.542220    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/63dace4c-7832-47dd-84ab-2c91483a5563-kube-api-access-d8lw2\" (UniqueName: \"kubernetes.io/projected/63dace4c-7832-47dd-84ab-2c91483a5563-kube-api-access-d8lw2\") pod \"63dace4c-7832-47dd-84ab-2c91483a5563\" (UID: \"63dace4c-7832-47dd-84ab-2c91483a5563\") "
Oct 30 13:20:22.544282 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:22.542301    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d65e47d2-4461-406a-9b71-bb3fac4262f0-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d65e47d2-4461-406a-9b71-bb3fac4262f0-tmp-dir\") pod \"d65e47d2-4461-406a-9b71-bb3fac4262f0\" (UID: \"d65e47d2-4461-406a-9b71-bb3fac4262f0\") "
Oct 30 13:20:22.544282 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:22.542351    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d65e47d2-4461-406a-9b71-bb3fac4262f0-dshm\" (UniqueName: \"kubernetes.io/empty-dir/d65e47d2-4461-406a-9b71-bb3fac4262f0-dshm\") pod \"d65e47d2-4461-406a-9b71-bb3fac4262f0\" (UID: \"d65e47d2-4461-406a-9b71-bb3fac4262f0\") "
Oct 30 13:20:22.544282 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:22.542433    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/63dace4c-7832-47dd-84ab-2c91483a5563-dshm\" (UniqueName: \"kubernetes.io/empty-dir/63dace4c-7832-47dd-84ab-2c91483a5563-dshm\") pod \"63dace4c-7832-47dd-84ab-2c91483a5563\" (UID: \"63dace4c-7832-47dd-84ab-2c91483a5563\") "
Oct 30 13:20:22.544282 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:22.542490    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/63dace4c-7832-47dd-84ab-2c91483a5563-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/63dace4c-7832-47dd-84ab-2c91483a5563-tmp-dir\") pod \"63dace4c-7832-47dd-84ab-2c91483a5563\" (UID: \"63dace4c-7832-47dd-84ab-2c91483a5563\") "
Oct 30 13:20:22.544282 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:22.542542    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d65e47d2-4461-406a-9b71-bb3fac4262f0-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d65e47d2-4461-406a-9b71-bb3fac4262f0-model-cache\") pod \"d65e47d2-4461-406a-9b71-bb3fac4262f0\" (UID: \"d65e47d2-4461-406a-9b71-bb3fac4262f0\") "
Oct 30 13:20:22.544282 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:22.542615    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/63dace4c-7832-47dd-84ab-2c91483a5563-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/63dace4c-7832-47dd-84ab-2c91483a5563-model-cache\") pod \"63dace4c-7832-47dd-84ab-2c91483a5563\" (UID: \"63dace4c-7832-47dd-84ab-2c91483a5563\") "
Oct 30 13:20:22.544282 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:22.542683    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/63dace4c-7832-47dd-84ab-2c91483a5563-home\" (UniqueName: \"kubernetes.io/empty-dir/63dace4c-7832-47dd-84ab-2c91483a5563-home\") pod \"63dace4c-7832-47dd-84ab-2c91483a5563\" (UID: \"63dace4c-7832-47dd-84ab-2c91483a5563\") "
Oct 30 13:20:22.544282 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:22.542759    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d65e47d2-4461-406a-9b71-bb3fac4262f0-kube-api-access-j2rvq\" (UniqueName: \"kubernetes.io/projected/d65e47d2-4461-406a-9b71-bb3fac4262f0-kube-api-access-j2rvq\") pod \"d65e47d2-4461-406a-9b71-bb3fac4262f0\" (UID: \"d65e47d2-4461-406a-9b71-bb3fac4262f0\") "
Oct 30 13:20:22.544282 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:22.542795    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/63dace4c-7832-47dd-84ab-2c91483a5563-tls-certs\" (UniqueName: \"kubernetes.io/secret/63dace4c-7832-47dd-84ab-2c91483a5563-tls-certs\") pod \"63dace4c-7832-47dd-84ab-2c91483a5563\" (UID: \"63dace4c-7832-47dd-84ab-2c91483a5563\") "
Oct 30 13:20:22.544282 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:22.542862    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d65e47d2-4461-406a-9b71-bb3fac4262f0-home\" (UniqueName: \"kubernetes.io/empty-dir/d65e47d2-4461-406a-9b71-bb3fac4262f0-home\") pod \"d65e47d2-4461-406a-9b71-bb3fac4262f0\" (UID: \"d65e47d2-4461-406a-9b71-bb3fac4262f0\") "
Oct 30 13:20:22.544282 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:22.542928    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d65e47d2-4461-406a-9b71-bb3fac4262f0-tls-certs\" (UniqueName: \"kubernetes.io/secret/d65e47d2-4461-406a-9b71-bb3fac4262f0-tls-certs\") pod \"d65e47d2-4461-406a-9b71-bb3fac4262f0\" (UID: \"d65e47d2-4461-406a-9b71-bb3fac4262f0\") "
Oct 30 13:20:22.563431 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:22.556395    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/63dace4c-7832-47dd-84ab-2c91483a5563-tmp-dir" pod "63dace4c-7832-47dd-84ab-2c91483a5563" (UID: "63dace4c-7832-47dd-84ab-2c91483a5563"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:20:22.563431 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:22.558818    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d65e47d2-4461-406a-9b71-bb3fac4262f0-tmp-dir" pod "d65e47d2-4461-406a-9b71-bb3fac4262f0" (UID: "d65e47d2-4461-406a-9b71-bb3fac4262f0"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:20:22.563431 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:22.558986    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/63dace4c-7832-47dd-84ab-2c91483a5563-home" pod "63dace4c-7832-47dd-84ab-2c91483a5563" (UID: "63dace4c-7832-47dd-84ab-2c91483a5563"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:20:22.563431 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:22.562348    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d65e47d2-4461-406a-9b71-bb3fac4262f0-model-cache" pod "d65e47d2-4461-406a-9b71-bb3fac4262f0" (UID: "d65e47d2-4461-406a-9b71-bb3fac4262f0"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:20:22.563431 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:22.562604    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/63dace4c-7832-47dd-84ab-2c91483a5563-model-cache" pod "63dace4c-7832-47dd-84ab-2c91483a5563" (UID: "63dace4c-7832-47dd-84ab-2c91483a5563"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:20:22.575342 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:22.575173    4943 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/63dace4c-7832-47dd-84ab-2c91483a5563-tmp-dir\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:20:22.575342 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:22.575213    4943 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d65e47d2-4461-406a-9b71-bb3fac4262f0-model-cache\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:20:22.575342 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:22.575227    4943 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/63dace4c-7832-47dd-84ab-2c91483a5563-model-cache\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:20:22.575342 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:22.575239    4943 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/63dace4c-7832-47dd-84ab-2c91483a5563-home\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:20:22.575342 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:22.575252    4943 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d65e47d2-4461-406a-9b71-bb3fac4262f0-tmp-dir\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:20:22.594352 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:22.593356    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d65e47d2-4461-406a-9b71-bb3fac4262f0-home" pod "d65e47d2-4461-406a-9b71-bb3fac4262f0" (UID: "d65e47d2-4461-406a-9b71-bb3fac4262f0"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:20:22.599293 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:22.599214    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/63dace4c-7832-47dd-84ab-2c91483a5563-dshm" pod "63dace4c-7832-47dd-84ab-2c91483a5563" (UID: "63dace4c-7832-47dd-84ab-2c91483a5563"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:20:22.599455 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:22.599304    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/63dace4c-7832-47dd-84ab-2c91483a5563-tls-certs" pod "63dace4c-7832-47dd-84ab-2c91483a5563" (UID: "63dace4c-7832-47dd-84ab-2c91483a5563"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 13:20:22.599676 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:22.599649    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d65e47d2-4461-406a-9b71-bb3fac4262f0-kube-api-access-j2rvq" pod "d65e47d2-4461-406a-9b71-bb3fac4262f0" (UID: "d65e47d2-4461-406a-9b71-bb3fac4262f0"). InnerVolumeSpecName "kube-api-access-j2rvq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:20:22.599921 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:22.599901    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/63dace4c-7832-47dd-84ab-2c91483a5563-kube-api-access-d8lw2" pod "63dace4c-7832-47dd-84ab-2c91483a5563" (UID: "63dace4c-7832-47dd-84ab-2c91483a5563"). InnerVolumeSpecName "kube-api-access-d8lw2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:20:22.611280 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:22.600897    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d65e47d2-4461-406a-9b71-bb3fac4262f0-dshm" pod "d65e47d2-4461-406a-9b71-bb3fac4262f0" (UID: "d65e47d2-4461-406a-9b71-bb3fac4262f0"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:20:22.619091 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:22.618559    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d65e47d2-4461-406a-9b71-bb3fac4262f0-tls-certs" pod "d65e47d2-4461-406a-9b71-bb3fac4262f0" (UID: "d65e47d2-4461-406a-9b71-bb3fac4262f0"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 13:20:22.677368 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:22.677323    4943 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j2rvq\" (UniqueName: \"kubernetes.io/projected/d65e47d2-4461-406a-9b71-bb3fac4262f0-kube-api-access-j2rvq\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:20:22.677368 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:22.677362    4943 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/63dace4c-7832-47dd-84ab-2c91483a5563-tls-certs\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:20:22.677669 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:22.677380    4943 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d65e47d2-4461-406a-9b71-bb3fac4262f0-home\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:20:22.677669 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:22.677393    4943 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d65e47d2-4461-406a-9b71-bb3fac4262f0-tls-certs\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:20:22.677669 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:22.677405    4943 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d8lw2\" (UniqueName: \"kubernetes.io/projected/63dace4c-7832-47dd-84ab-2c91483a5563-kube-api-access-d8lw2\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:20:22.677669 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:22.677415    4943 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d65e47d2-4461-406a-9b71-bb3fac4262f0-dshm\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:20:22.677669 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:22.677426    4943 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/63dace4c-7832-47dd-84ab-2c91483a5563-dshm\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:20:22.973768 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:22.973681    4943 generic.go:356] "Generic (PLEG): container finished" podID="d65e47d2-4461-406a-9b71-bb3fac4262f0" containerID="295b4280acd1ea46140fd906e618c2bee2291632e55178b166967be88fa38ce6" exitCode=0
Oct 30 13:20:22.973768 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:22.973751    4943 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-d0a18c/distinct-2-w3-d0a18c-kserve-85f8ff4846-rc44r"
Oct 30 13:20:22.974017 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:22.973763    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-d0a18c/distinct-2-w3-d0a18c-kserve-85f8ff4846-rc44r" event={"ID":"d65e47d2-4461-406a-9b71-bb3fac4262f0","Type":"ContainerDied","Data":"295b4280acd1ea46140fd906e618c2bee2291632e55178b166967be88fa38ce6"}
Oct 30 13:20:22.974017 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:22.973804    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-d0a18c/distinct-2-w3-d0a18c-kserve-85f8ff4846-rc44r" event={"ID":"d65e47d2-4461-406a-9b71-bb3fac4262f0","Type":"ContainerDied","Data":"60b0b5e7e17beaa1908e291825b35e65a78a7e7208c49375be251f2001742c8c"}
Oct 30 13:20:22.974017 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:22.973820    4943 scope.go:122] "RemoveContainer" containerID="295b4280acd1ea46140fd906e618c2bee2291632e55178b166967be88fa38ce6"
Oct 30 13:20:22.975422 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:22.975400    4943 generic.go:356] "Generic (PLEG): container finished" podID="63dace4c-7832-47dd-84ab-2c91483a5563" containerID="adeb6ab24fe6238092db30053af4cd8cc74a617b87301baf954b8382f346cd18" exitCode=0
Oct 30 13:20:22.975536 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:22.975455    4943 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-d0a18c/distinct-w3-d0a18c-kserve-86479f6bbf-5zcqq"
Oct 30 13:20:22.975536 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:22.975484    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-d0a18c/distinct-w3-d0a18c-kserve-86479f6bbf-5zcqq" event={"ID":"63dace4c-7832-47dd-84ab-2c91483a5563","Type":"ContainerDied","Data":"adeb6ab24fe6238092db30053af4cd8cc74a617b87301baf954b8382f346cd18"}
Oct 30 13:20:22.975648 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:22.975539    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-d0a18c/distinct-w3-d0a18c-kserve-86479f6bbf-5zcqq" event={"ID":"63dace4c-7832-47dd-84ab-2c91483a5563","Type":"ContainerDied","Data":"21b6086af0982832b49a290560002b8b51df555345fb1df9e6a20f727195e6a8"}
Oct 30 13:20:22.992407 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:22.992383    4943 scope.go:122] "RemoveContainer" containerID="295b4280acd1ea46140fd906e618c2bee2291632e55178b166967be88fa38ce6"
Oct 30 13:20:22.992722 ip-10-0-142-84 kubenswrapper[4943]: E1030 13:20:22.992705    4943 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"295b4280acd1ea46140fd906e618c2bee2291632e55178b166967be88fa38ce6\": container with ID starting with 295b4280acd1ea46140fd906e618c2bee2291632e55178b166967be88fa38ce6 not found: ID does not exist" containerID="295b4280acd1ea46140fd906e618c2bee2291632e55178b166967be88fa38ce6"
Oct 30 13:20:22.992780 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:22.992728    4943 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"295b4280acd1ea46140fd906e618c2bee2291632e55178b166967be88fa38ce6"} err="failed to get container status \"295b4280acd1ea46140fd906e618c2bee2291632e55178b166967be88fa38ce6\": rpc error: code = NotFound desc = could not find container \"295b4280acd1ea46140fd906e618c2bee2291632e55178b166967be88fa38ce6\": container with ID starting with 295b4280acd1ea46140fd906e618c2bee2291632e55178b166967be88fa38ce6 not found: ID does not exist"
Oct 30 13:20:22.992780 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:22.992744    4943 scope.go:122] "RemoveContainer" containerID="adeb6ab24fe6238092db30053af4cd8cc74a617b87301baf954b8382f346cd18"
Oct 30 13:20:23.003332 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:23.003307    4943 scope.go:122] "RemoveContainer" containerID="adeb6ab24fe6238092db30053af4cd8cc74a617b87301baf954b8382f346cd18"
Oct 30 13:20:23.003651 ip-10-0-142-84 kubenswrapper[4943]: E1030 13:20:23.003628    4943 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"adeb6ab24fe6238092db30053af4cd8cc74a617b87301baf954b8382f346cd18\": container with ID starting with adeb6ab24fe6238092db30053af4cd8cc74a617b87301baf954b8382f346cd18 not found: ID does not exist" containerID="adeb6ab24fe6238092db30053af4cd8cc74a617b87301baf954b8382f346cd18"
Oct 30 13:20:23.003723 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:23.003656    4943 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"adeb6ab24fe6238092db30053af4cd8cc74a617b87301baf954b8382f346cd18"} err="failed to get container status \"adeb6ab24fe6238092db30053af4cd8cc74a617b87301baf954b8382f346cd18\": rpc error: code = NotFound desc = could not find container \"adeb6ab24fe6238092db30053af4cd8cc74a617b87301baf954b8382f346cd18\": container with ID starting with adeb6ab24fe6238092db30053af4cd8cc74a617b87301baf954b8382f346cd18 not found: ID does not exist"
Oct 30 13:20:23.042316 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:23.042294    4943 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w3-d0a18c/distinct-w3-d0a18c-kserve-86479f6bbf-5zcqq"]
Oct 30 13:20:23.048699 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:23.048669    4943 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w3-d0a18c/distinct-w3-d0a18c-kserve-86479f6bbf-5zcqq"]
Oct 30 13:20:23.089971 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:23.089951    4943 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w3-d0a18c/distinct-2-w3-d0a18c-kserve-85f8ff4846-rc44r"]
Oct 30 13:20:23.148290 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:23.146860    4943 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w3-d0a18c/distinct-2-w3-d0a18c-kserve-85f8ff4846-rc44r"]
Oct 30 13:20:24.351584 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:24.351541    4943 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="63dace4c-7832-47dd-84ab-2c91483a5563" path="/var/lib/kubelet/pods/63dace4c-7832-47dd-84ab-2c91483a5563/volumes"
Oct 30 13:20:24.352186 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:24.352162    4943 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d65e47d2-4461-406a-9b71-bb3fac4262f0" path="/var/lib/kubelet/pods/d65e47d2-4461-406a-9b71-bb3fac4262f0/volumes"
Oct 30 13:20:26.017956 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:26.017924    4943 generic.go:356] "Generic (PLEG): container finished" podID="0b3caf04-1a9d-44ce-8d80-3335b28a36ab" containerID="82312698550d72f0e0cc6943d93c514b690a9e6ebc12f5d447d6cb49a5ba1d5b" exitCode=0
Oct 30 13:20:26.018406 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:26.018001    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-58kq2" event={"ID":"0b3caf04-1a9d-44ce-8d80-3335b28a36ab","Type":"ContainerDied","Data":"82312698550d72f0e0cc6943d93c514b690a9e6ebc12f5d447d6cb49a5ba1d5b"}
Oct 30 13:20:26.019932 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:26.019907    4943 generic.go:356] "Generic (PLEG): container finished" podID="663524af-aa09-44fe-8186-654cb9668d27" containerID="a7aab891628438e6299302e8f696c682fa173c6902aa23e763ea7aa6424ad66a" exitCode=0
Oct 30 13:20:26.020300 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:26.020280    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-f55w2" event={"ID":"663524af-aa09-44fe-8186-654cb9668d27","Type":"ContainerDied","Data":"a7aab891628438e6299302e8f696c682fa173c6902aa23e763ea7aa6424ad66a"}
Oct 30 13:20:26.022373 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:26.022351    4943 generic.go:356] "Generic (PLEG): container finished" podID="dbc82b1c-1c45-4263-b8ee-090fbecff086" containerID="08c663942fb9d9c7fa0105b756682ffc83a25223f84ca89191e85c5f7298e19c" exitCode=0
Oct 30 13:20:26.022492 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:26.022390    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-glf25" event={"ID":"dbc82b1c-1c45-4263-b8ee-090fbecff086","Type":"ContainerDied","Data":"08c663942fb9d9c7fa0105b756682ffc83a25223f84ca89191e85c5f7298e19c"}
Oct 30 13:20:26.104133 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:26.103637    4943 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-58kq2"
Oct 30 13:20:26.160146 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:26.160114    4943 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-glf25"
Oct 30 13:20:26.174945 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:26.174924    4943 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-f55w2"
Oct 30 13:20:26.238146 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:26.238112    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0b3caf04-1a9d-44ce-8d80-3335b28a36ab-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0b3caf04-1a9d-44ce-8d80-3335b28a36ab-credential-socket\") pod \"0b3caf04-1a9d-44ce-8d80-3335b28a36ab\" (UID: \"0b3caf04-1a9d-44ce-8d80-3335b28a36ab\") "
Oct 30 13:20:26.238302 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:26.238226    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0b3caf04-1a9d-44ce-8d80-3335b28a36ab-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0b3caf04-1a9d-44ce-8d80-3335b28a36ab-istiod-ca-cert\") pod \"0b3caf04-1a9d-44ce-8d80-3335b28a36ab\" (UID: \"0b3caf04-1a9d-44ce-8d80-3335b28a36ab\") "
Oct 30 13:20:26.238392 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:26.238375    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/0b3caf04-1a9d-44ce-8d80-3335b28a36ab-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0b3caf04-1a9d-44ce-8d80-3335b28a36ab-istio-podinfo\") pod \"0b3caf04-1a9d-44ce-8d80-3335b28a36ab\" (UID: \"0b3caf04-1a9d-44ce-8d80-3335b28a36ab\") "
Oct 30 13:20:26.238443 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:26.238411    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0b3caf04-1a9d-44ce-8d80-3335b28a36ab-istio-token\" (UniqueName: \"kubernetes.io/projected/0b3caf04-1a9d-44ce-8d80-3335b28a36ab-istio-token\") pod \"0b3caf04-1a9d-44ce-8d80-3335b28a36ab\" (UID: \"0b3caf04-1a9d-44ce-8d80-3335b28a36ab\") "
Oct 30 13:20:26.238496 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:26.238469    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0b3caf04-1a9d-44ce-8d80-3335b28a36ab-kube-api-access-nxchf\" (UniqueName: \"kubernetes.io/projected/0b3caf04-1a9d-44ce-8d80-3335b28a36ab-kube-api-access-nxchf\") pod \"0b3caf04-1a9d-44ce-8d80-3335b28a36ab\" (UID: \"0b3caf04-1a9d-44ce-8d80-3335b28a36ab\") "
Oct 30 13:20:26.238551 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:26.238497    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0b3caf04-1a9d-44ce-8d80-3335b28a36ab-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0b3caf04-1a9d-44ce-8d80-3335b28a36ab-istio-envoy\") pod \"0b3caf04-1a9d-44ce-8d80-3335b28a36ab\" (UID: \"0b3caf04-1a9d-44ce-8d80-3335b28a36ab\") "
Oct 30 13:20:26.238551 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:26.238407    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0b3caf04-1a9d-44ce-8d80-3335b28a36ab-credential-socket" pod "0b3caf04-1a9d-44ce-8d80-3335b28a36ab" (UID: "0b3caf04-1a9d-44ce-8d80-3335b28a36ab"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:20:26.238670 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:26.238605    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0b3caf04-1a9d-44ce-8d80-3335b28a36ab-istiod-ca-cert" pod "0b3caf04-1a9d-44ce-8d80-3335b28a36ab" (UID: "0b3caf04-1a9d-44ce-8d80-3335b28a36ab"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 13:20:26.238670 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:26.238597    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0b3caf04-1a9d-44ce-8d80-3335b28a36ab-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0b3caf04-1a9d-44ce-8d80-3335b28a36ab-workload-certs\") pod \"0b3caf04-1a9d-44ce-8d80-3335b28a36ab\" (UID: \"0b3caf04-1a9d-44ce-8d80-3335b28a36ab\") "
Oct 30 13:20:26.238784 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:26.238759    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0b3caf04-1a9d-44ce-8d80-3335b28a36ab-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0b3caf04-1a9d-44ce-8d80-3335b28a36ab-workload-socket\") pod \"0b3caf04-1a9d-44ce-8d80-3335b28a36ab\" (UID: \"0b3caf04-1a9d-44ce-8d80-3335b28a36ab\") "
Oct 30 13:20:26.238975 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:26.238955    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0b3caf04-1a9d-44ce-8d80-3335b28a36ab-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0b3caf04-1a9d-44ce-8d80-3335b28a36ab-istio-data\") pod \"0b3caf04-1a9d-44ce-8d80-3335b28a36ab\" (UID: \"0b3caf04-1a9d-44ce-8d80-3335b28a36ab\") "
Oct 30 13:20:26.239233 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:26.239060    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0b3caf04-1a9d-44ce-8d80-3335b28a36ab-workload-socket" pod "0b3caf04-1a9d-44ce-8d80-3335b28a36ab" (UID: "0b3caf04-1a9d-44ce-8d80-3335b28a36ab"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:20:26.239350 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:26.239235    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0b3caf04-1a9d-44ce-8d80-3335b28a36ab-istio-data" pod "0b3caf04-1a9d-44ce-8d80-3335b28a36ab" (UID: "0b3caf04-1a9d-44ce-8d80-3335b28a36ab"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:20:26.239350 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:26.239329    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0b3caf04-1a9d-44ce-8d80-3335b28a36ab-workload-certs" pod "0b3caf04-1a9d-44ce-8d80-3335b28a36ab" (UID: "0b3caf04-1a9d-44ce-8d80-3335b28a36ab"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:20:26.240095 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:26.240067    4943 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0b3caf04-1a9d-44ce-8d80-3335b28a36ab-workload-certs\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:20:26.240174 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:26.240102    4943 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0b3caf04-1a9d-44ce-8d80-3335b28a36ab-workload-socket\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:20:26.240174 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:26.240117    4943 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0b3caf04-1a9d-44ce-8d80-3335b28a36ab-istio-data\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:20:26.240174 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:26.240132    4943 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0b3caf04-1a9d-44ce-8d80-3335b28a36ab-credential-socket\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:20:26.240174 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:26.240146    4943 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0b3caf04-1a9d-44ce-8d80-3335b28a36ab-istiod-ca-cert\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:20:26.241023 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:26.240996    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/0b3caf04-1a9d-44ce-8d80-3335b28a36ab-istio-podinfo" pod "0b3caf04-1a9d-44ce-8d80-3335b28a36ab" (UID: "0b3caf04-1a9d-44ce-8d80-3335b28a36ab"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 13:20:26.241106 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:26.241086    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0b3caf04-1a9d-44ce-8d80-3335b28a36ab-istio-envoy" pod "0b3caf04-1a9d-44ce-8d80-3335b28a36ab" (UID: "0b3caf04-1a9d-44ce-8d80-3335b28a36ab"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:20:26.241148 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:26.241112    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0b3caf04-1a9d-44ce-8d80-3335b28a36ab-kube-api-access-nxchf" pod "0b3caf04-1a9d-44ce-8d80-3335b28a36ab" (UID: "0b3caf04-1a9d-44ce-8d80-3335b28a36ab"). InnerVolumeSpecName "kube-api-access-nxchf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:20:26.242489 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:26.242470    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0b3caf04-1a9d-44ce-8d80-3335b28a36ab-istio-token" pod "0b3caf04-1a9d-44ce-8d80-3335b28a36ab" (UID: "0b3caf04-1a9d-44ce-8d80-3335b28a36ab"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:20:26.341177 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:26.341090    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/dbc82b1c-1c45-4263-b8ee-090fbecff086-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dbc82b1c-1c45-4263-b8ee-090fbecff086-istio-data\") pod \"dbc82b1c-1c45-4263-b8ee-090fbecff086\" (UID: \"dbc82b1c-1c45-4263-b8ee-090fbecff086\") "
Oct 30 13:20:26.341355 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:26.341192    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/663524af-aa09-44fe-8186-654cb9668d27-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/663524af-aa09-44fe-8186-654cb9668d27-workload-socket\") pod \"663524af-aa09-44fe-8186-654cb9668d27\" (UID: \"663524af-aa09-44fe-8186-654cb9668d27\") "
Oct 30 13:20:26.341355 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:26.341235    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/dbc82b1c-1c45-4263-b8ee-090fbecff086-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dbc82b1c-1c45-4263-b8ee-090fbecff086-istio-podinfo\") pod \"dbc82b1c-1c45-4263-b8ee-090fbecff086\" (UID: \"dbc82b1c-1c45-4263-b8ee-090fbecff086\") "
Oct 30 13:20:26.341355 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:26.341286    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/663524af-aa09-44fe-8186-654cb9668d27-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/663524af-aa09-44fe-8186-654cb9668d27-istio-envoy\") pod \"663524af-aa09-44fe-8186-654cb9668d27\" (UID: \"663524af-aa09-44fe-8186-654cb9668d27\") "
Oct 30 13:20:26.341540 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:26.341368    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/663524af-aa09-44fe-8186-654cb9668d27-kube-api-access-jzvdf\" (UniqueName: \"kubernetes.io/projected/663524af-aa09-44fe-8186-654cb9668d27-kube-api-access-jzvdf\") pod \"663524af-aa09-44fe-8186-654cb9668d27\" (UID: \"663524af-aa09-44fe-8186-654cb9668d27\") "
Oct 30 13:20:26.341540 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:26.341416    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/dbc82b1c-1c45-4263-b8ee-090fbecff086-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dbc82b1c-1c45-4263-b8ee-090fbecff086-workload-certs\") pod \"dbc82b1c-1c45-4263-b8ee-090fbecff086\" (UID: \"dbc82b1c-1c45-4263-b8ee-090fbecff086\") "
Oct 30 13:20:26.341540 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:26.341452    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/663524af-aa09-44fe-8186-654cb9668d27-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/663524af-aa09-44fe-8186-654cb9668d27-workload-certs\") pod \"663524af-aa09-44fe-8186-654cb9668d27\" (UID: \"663524af-aa09-44fe-8186-654cb9668d27\") "
Oct 30 13:20:26.341540 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:26.341451    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dbc82b1c-1c45-4263-b8ee-090fbecff086-istio-data" pod "dbc82b1c-1c45-4263-b8ee-090fbecff086" (UID: "dbc82b1c-1c45-4263-b8ee-090fbecff086"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:20:26.341540 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:26.341497    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/663524af-aa09-44fe-8186-654cb9668d27-workload-socket" pod "663524af-aa09-44fe-8186-654cb9668d27" (UID: "663524af-aa09-44fe-8186-654cb9668d27"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:20:26.341793 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:26.341544    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/663524af-aa09-44fe-8186-654cb9668d27-istio-token\" (UniqueName: \"kubernetes.io/projected/663524af-aa09-44fe-8186-654cb9668d27-istio-token\") pod \"663524af-aa09-44fe-8186-654cb9668d27\" (UID: \"663524af-aa09-44fe-8186-654cb9668d27\") "
Oct 30 13:20:26.341793 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:26.341584    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/663524af-aa09-44fe-8186-654cb9668d27-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/663524af-aa09-44fe-8186-654cb9668d27-istio-podinfo\") pod \"663524af-aa09-44fe-8186-654cb9668d27\" (UID: \"663524af-aa09-44fe-8186-654cb9668d27\") "
Oct 30 13:20:26.341793 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:26.341623    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/663524af-aa09-44fe-8186-654cb9668d27-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/663524af-aa09-44fe-8186-654cb9668d27-istio-data\") pod \"663524af-aa09-44fe-8186-654cb9668d27\" (UID: \"663524af-aa09-44fe-8186-654cb9668d27\") "
Oct 30 13:20:26.341793 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:26.341702    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/dbc82b1c-1c45-4263-b8ee-090fbecff086-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dbc82b1c-1c45-4263-b8ee-090fbecff086-credential-socket\") pod \"dbc82b1c-1c45-4263-b8ee-090fbecff086\" (UID: \"dbc82b1c-1c45-4263-b8ee-090fbecff086\") "
Oct 30 13:20:26.341793 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:26.341732    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/663524af-aa09-44fe-8186-654cb9668d27-workload-certs" pod "663524af-aa09-44fe-8186-654cb9668d27" (UID: "663524af-aa09-44fe-8186-654cb9668d27"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:20:26.341793 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:26.341786    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/dbc82b1c-1c45-4263-b8ee-090fbecff086-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dbc82b1c-1c45-4263-b8ee-090fbecff086-workload-socket\") pod \"dbc82b1c-1c45-4263-b8ee-090fbecff086\" (UID: \"dbc82b1c-1c45-4263-b8ee-090fbecff086\") "
Oct 30 13:20:26.342100 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:26.341814    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/dbc82b1c-1c45-4263-b8ee-090fbecff086-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dbc82b1c-1c45-4263-b8ee-090fbecff086-istio-envoy\") pod \"dbc82b1c-1c45-4263-b8ee-090fbecff086\" (UID: \"dbc82b1c-1c45-4263-b8ee-090fbecff086\") "
Oct 30 13:20:26.342100 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:26.341860    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/663524af-aa09-44fe-8186-654cb9668d27-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/663524af-aa09-44fe-8186-654cb9668d27-istiod-ca-cert\") pod \"663524af-aa09-44fe-8186-654cb9668d27\" (UID: \"663524af-aa09-44fe-8186-654cb9668d27\") "
Oct 30 13:20:26.342100 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:26.341883    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dbc82b1c-1c45-4263-b8ee-090fbecff086-workload-certs" pod "dbc82b1c-1c45-4263-b8ee-090fbecff086" (UID: "dbc82b1c-1c45-4263-b8ee-090fbecff086"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:20:26.342100 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:26.341909    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/dbc82b1c-1c45-4263-b8ee-090fbecff086-kube-api-access-f2jml\" (UniqueName: \"kubernetes.io/projected/dbc82b1c-1c45-4263-b8ee-090fbecff086-kube-api-access-f2jml\") pod \"dbc82b1c-1c45-4263-b8ee-090fbecff086\" (UID: \"dbc82b1c-1c45-4263-b8ee-090fbecff086\") "
Oct 30 13:20:26.342100 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:26.341946    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/663524af-aa09-44fe-8186-654cb9668d27-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/663524af-aa09-44fe-8186-654cb9668d27-credential-socket\") pod \"663524af-aa09-44fe-8186-654cb9668d27\" (UID: \"663524af-aa09-44fe-8186-654cb9668d27\") "
Oct 30 13:20:26.342100 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:26.341981    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/dbc82b1c-1c45-4263-b8ee-090fbecff086-istio-token\" (UniqueName: \"kubernetes.io/projected/dbc82b1c-1c45-4263-b8ee-090fbecff086-istio-token\") pod \"dbc82b1c-1c45-4263-b8ee-090fbecff086\" (UID: \"dbc82b1c-1c45-4263-b8ee-090fbecff086\") "
Oct 30 13:20:26.342100 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:26.342024    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/dbc82b1c-1c45-4263-b8ee-090fbecff086-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dbc82b1c-1c45-4263-b8ee-090fbecff086-istiod-ca-cert\") pod \"dbc82b1c-1c45-4263-b8ee-090fbecff086\" (UID: \"dbc82b1c-1c45-4263-b8ee-090fbecff086\") "
Oct 30 13:20:26.342100 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:26.342083    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dbc82b1c-1c45-4263-b8ee-090fbecff086-credential-socket" pod "dbc82b1c-1c45-4263-b8ee-090fbecff086" (UID: "dbc82b1c-1c45-4263-b8ee-090fbecff086"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:20:26.342652 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:26.342628    4943 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/663524af-aa09-44fe-8186-654cb9668d27-workload-socket\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:20:26.342729 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:26.342661    4943 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dbc82b1c-1c45-4263-b8ee-090fbecff086-workload-certs\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:20:26.342729 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:26.342677    4943 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/663524af-aa09-44fe-8186-654cb9668d27-workload-certs\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:20:26.342729 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:26.342690    4943 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dbc82b1c-1c45-4263-b8ee-090fbecff086-credential-socket\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:20:26.342729 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:26.342704    4943 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0b3caf04-1a9d-44ce-8d80-3335b28a36ab-istio-podinfo\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:20:26.342729 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:26.342718    4943 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0b3caf04-1a9d-44ce-8d80-3335b28a36ab-istio-token\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:20:26.342955 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:26.342731    4943 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nxchf\" (UniqueName: \"kubernetes.io/projected/0b3caf04-1a9d-44ce-8d80-3335b28a36ab-kube-api-access-nxchf\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:20:26.342955 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:26.342744    4943 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0b3caf04-1a9d-44ce-8d80-3335b28a36ab-istio-envoy\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:20:26.342955 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:26.342759    4943 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dbc82b1c-1c45-4263-b8ee-090fbecff086-istio-data\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:20:26.343084 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:26.342992    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/663524af-aa09-44fe-8186-654cb9668d27-istiod-ca-cert" pod "663524af-aa09-44fe-8186-654cb9668d27" (UID: "663524af-aa09-44fe-8186-654cb9668d27"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 13:20:26.343084 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:26.343059    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dbc82b1c-1c45-4263-b8ee-090fbecff086-istiod-ca-cert" pod "dbc82b1c-1c45-4263-b8ee-090fbecff086" (UID: "dbc82b1c-1c45-4263-b8ee-090fbecff086"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 13:20:26.343304 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:26.343279    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dbc82b1c-1c45-4263-b8ee-090fbecff086-workload-socket" pod "dbc82b1c-1c45-4263-b8ee-090fbecff086" (UID: "dbc82b1c-1c45-4263-b8ee-090fbecff086"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:20:26.343784 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:26.343757    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/663524af-aa09-44fe-8186-654cb9668d27-istio-data" pod "663524af-aa09-44fe-8186-654cb9668d27" (UID: "663524af-aa09-44fe-8186-654cb9668d27"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:20:26.343998 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:26.343974    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/663524af-aa09-44fe-8186-654cb9668d27-credential-socket" pod "663524af-aa09-44fe-8186-654cb9668d27" (UID: "663524af-aa09-44fe-8186-654cb9668d27"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:20:26.344348 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:26.344321    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/663524af-aa09-44fe-8186-654cb9668d27-istio-envoy" pod "663524af-aa09-44fe-8186-654cb9668d27" (UID: "663524af-aa09-44fe-8186-654cb9668d27"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:20:26.345337 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:26.345307    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/663524af-aa09-44fe-8186-654cb9668d27-kube-api-access-jzvdf" pod "663524af-aa09-44fe-8186-654cb9668d27" (UID: "663524af-aa09-44fe-8186-654cb9668d27"). InnerVolumeSpecName "kube-api-access-jzvdf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:20:26.347244 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:26.347210    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dbc82b1c-1c45-4263-b8ee-090fbecff086-kube-api-access-f2jml" pod "dbc82b1c-1c45-4263-b8ee-090fbecff086" (UID: "dbc82b1c-1c45-4263-b8ee-090fbecff086"). InnerVolumeSpecName "kube-api-access-f2jml". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:20:26.347513 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:26.347490    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/663524af-aa09-44fe-8186-654cb9668d27-istio-token" pod "663524af-aa09-44fe-8186-654cb9668d27" (UID: "663524af-aa09-44fe-8186-654cb9668d27"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:20:26.347752 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:26.347697    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/dbc82b1c-1c45-4263-b8ee-090fbecff086-istio-podinfo" pod "dbc82b1c-1c45-4263-b8ee-090fbecff086" (UID: "dbc82b1c-1c45-4263-b8ee-090fbecff086"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 13:20:26.347752 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:26.347719    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/663524af-aa09-44fe-8186-654cb9668d27-istio-podinfo" pod "663524af-aa09-44fe-8186-654cb9668d27" (UID: "663524af-aa09-44fe-8186-654cb9668d27"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 13:20:26.348221 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:26.348190    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dbc82b1c-1c45-4263-b8ee-090fbecff086-istio-envoy" pod "dbc82b1c-1c45-4263-b8ee-090fbecff086" (UID: "dbc82b1c-1c45-4263-b8ee-090fbecff086"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:20:26.350142 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:26.350114    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dbc82b1c-1c45-4263-b8ee-090fbecff086-istio-token" pod "dbc82b1c-1c45-4263-b8ee-090fbecff086" (UID: "dbc82b1c-1c45-4263-b8ee-090fbecff086"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:20:26.444769 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:26.444555    4943 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f2jml\" (UniqueName: \"kubernetes.io/projected/dbc82b1c-1c45-4263-b8ee-090fbecff086-kube-api-access-f2jml\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:20:26.444769 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:26.444589    4943 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/663524af-aa09-44fe-8186-654cb9668d27-credential-socket\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:20:26.444769 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:26.444604    4943 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dbc82b1c-1c45-4263-b8ee-090fbecff086-istio-token\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:20:26.444769 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:26.444619    4943 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dbc82b1c-1c45-4263-b8ee-090fbecff086-istiod-ca-cert\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:20:26.444769 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:26.444632    4943 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dbc82b1c-1c45-4263-b8ee-090fbecff086-istio-podinfo\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:20:26.444769 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:26.444664    4943 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/663524af-aa09-44fe-8186-654cb9668d27-istio-envoy\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:20:26.444769 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:26.444690    4943 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jzvdf\" (UniqueName: \"kubernetes.io/projected/663524af-aa09-44fe-8186-654cb9668d27-kube-api-access-jzvdf\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:20:26.444769 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:26.444708    4943 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/663524af-aa09-44fe-8186-654cb9668d27-istio-token\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:20:26.444769 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:26.444725    4943 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/663524af-aa09-44fe-8186-654cb9668d27-istio-podinfo\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:20:26.444769 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:26.444740    4943 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/663524af-aa09-44fe-8186-654cb9668d27-istio-data\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:20:26.444769 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:26.444755    4943 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dbc82b1c-1c45-4263-b8ee-090fbecff086-workload-socket\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:20:26.444769 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:26.444768    4943 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dbc82b1c-1c45-4263-b8ee-090fbecff086-istio-envoy\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:20:26.444769 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:26.444782    4943 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/663524af-aa09-44fe-8186-654cb9668d27-istiod-ca-cert\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:20:27.033657 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:27.033628    4943 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-f55w2"
Oct 30 13:20:27.034049 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:27.033653    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-f55w2" event={"ID":"663524af-aa09-44fe-8186-654cb9668d27","Type":"ContainerDied","Data":"35eff802ed42ee69a610841e96391731fddf9bd67b51db2310118f3539f87166"}
Oct 30 13:20:27.034049 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:27.033746    4943 scope.go:122] "RemoveContainer" containerID="a7aab891628438e6299302e8f696c682fa173c6902aa23e763ea7aa6424ad66a"
Oct 30 13:20:27.035624 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:27.035601    4943 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-glf25"
Oct 30 13:20:27.035624 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:27.035613    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-glf25" event={"ID":"dbc82b1c-1c45-4263-b8ee-090fbecff086","Type":"ContainerDied","Data":"e5ffcccf9ca8b7194cd890fe37f69da356a7be73f7ac72f5e477b8608f953777"}
Oct 30 13:20:27.037413 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:27.037389    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-58kq2" event={"ID":"0b3caf04-1a9d-44ce-8d80-3335b28a36ab","Type":"ContainerDied","Data":"165834ba1cc2100fdd411cacd92ddd10641d217fbd15c3bd409cd3aaa8e2eca3"}
Oct 30 13:20:27.037536 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:27.037434    4943 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-58kq2"
Oct 30 13:20:27.046436 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:27.046414    4943 scope.go:122] "RemoveContainer" containerID="08c663942fb9d9c7fa0105b756682ffc83a25223f84ca89191e85c5f7298e19c"
Oct 30 13:20:27.056578 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:27.056452    4943 scope.go:122] "RemoveContainer" containerID="82312698550d72f0e0cc6943d93c514b690a9e6ebc12f5d447d6cb49a5ba1d5b"
Oct 30 13:20:27.099688 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:27.099660    4943 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-glf25"]
Oct 30 13:20:27.113455 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:27.113429    4943 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-glf25"]
Oct 30 13:20:27.143469 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:27.143440    4943 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w4-88f6a4/distinct-2-w4-88f6a4-kserve-7b8f7bd7dc-5bt68"
Oct 30 13:20:27.145033 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:27.145008    4943 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-f55w2"]
Oct 30 13:20:27.149095 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:27.149074    4943 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-f55w2"]
Oct 30 13:20:27.156332 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:27.156311    4943 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w4-88f6a4/distinct-2-w4-88f6a4-kserve-7b8f7bd7dc-5bt68"
Oct 30 13:20:27.185303 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:27.185278    4943 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-58kq2"]
Oct 30 13:20:27.195993 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:27.195968    4943 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-58kq2"]
Oct 30 13:20:27.790374 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:27.790350    4943 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-c98s9"
Oct 30 13:20:27.795363 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:27.795336    4943 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-nbdn6"
Oct 30 13:20:27.849735 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:27.849711    4943 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-zmddf"
Oct 30 13:20:27.963420 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:27.963386    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/094f15a2-03fc-4322-9bcb-c190f5a6d8c4-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/094f15a2-03fc-4322-9bcb-c190f5a6d8c4-istio-data\") pod \"094f15a2-03fc-4322-9bcb-c190f5a6d8c4\" (UID: \"094f15a2-03fc-4322-9bcb-c190f5a6d8c4\") "
Oct 30 13:20:27.963610 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:27.963475    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/78fecfa2-495c-4690-bd0e-08e0a4213ade-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/78fecfa2-495c-4690-bd0e-08e0a4213ade-credential-socket\") pod \"78fecfa2-495c-4690-bd0e-08e0a4213ade\" (UID: \"78fecfa2-495c-4690-bd0e-08e0a4213ade\") "
Oct 30 13:20:27.963610 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:27.963508    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/094f15a2-03fc-4322-9bcb-c190f5a6d8c4-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/094f15a2-03fc-4322-9bcb-c190f5a6d8c4-workload-socket\") pod \"094f15a2-03fc-4322-9bcb-c190f5a6d8c4\" (UID: \"094f15a2-03fc-4322-9bcb-c190f5a6d8c4\") "
Oct 30 13:20:27.963610 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:27.963536    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/094f15a2-03fc-4322-9bcb-c190f5a6d8c4-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/094f15a2-03fc-4322-9bcb-c190f5a6d8c4-istio-podinfo\") pod \"094f15a2-03fc-4322-9bcb-c190f5a6d8c4\" (UID: \"094f15a2-03fc-4322-9bcb-c190f5a6d8c4\") "
Oct 30 13:20:27.963610 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:27.963568    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/78fecfa2-495c-4690-bd0e-08e0a4213ade-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/78fecfa2-495c-4690-bd0e-08e0a4213ade-istiod-ca-cert\") pod \"78fecfa2-495c-4690-bd0e-08e0a4213ade\" (UID: \"78fecfa2-495c-4690-bd0e-08e0a4213ade\") "
Oct 30 13:20:27.963610 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:27.963600    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7088cc53-0ec6-43be-9669-075ca0ad2fa1-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7088cc53-0ec6-43be-9669-075ca0ad2fa1-workload-certs\") pod \"7088cc53-0ec6-43be-9669-075ca0ad2fa1\" (UID: \"7088cc53-0ec6-43be-9669-075ca0ad2fa1\") "
Oct 30 13:20:27.963901 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:27.963654    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/094f15a2-03fc-4322-9bcb-c190f5a6d8c4-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/094f15a2-03fc-4322-9bcb-c190f5a6d8c4-istiod-ca-cert\") pod \"094f15a2-03fc-4322-9bcb-c190f5a6d8c4\" (UID: \"094f15a2-03fc-4322-9bcb-c190f5a6d8c4\") "
Oct 30 13:20:27.963901 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:27.963684    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/78fecfa2-495c-4690-bd0e-08e0a4213ade-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/78fecfa2-495c-4690-bd0e-08e0a4213ade-istio-podinfo\") pod \"78fecfa2-495c-4690-bd0e-08e0a4213ade\" (UID: \"78fecfa2-495c-4690-bd0e-08e0a4213ade\") "
Oct 30 13:20:27.963901 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:27.963719    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7088cc53-0ec6-43be-9669-075ca0ad2fa1-kube-api-access-m2dgf\" (UniqueName: \"kubernetes.io/projected/7088cc53-0ec6-43be-9669-075ca0ad2fa1-kube-api-access-m2dgf\") pod \"7088cc53-0ec6-43be-9669-075ca0ad2fa1\" (UID: \"7088cc53-0ec6-43be-9669-075ca0ad2fa1\") "
Oct 30 13:20:27.963901 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:27.963723    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/094f15a2-03fc-4322-9bcb-c190f5a6d8c4-istio-data" pod "094f15a2-03fc-4322-9bcb-c190f5a6d8c4" (UID: "094f15a2-03fc-4322-9bcb-c190f5a6d8c4"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:20:27.963901 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:27.963750    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7088cc53-0ec6-43be-9669-075ca0ad2fa1-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7088cc53-0ec6-43be-9669-075ca0ad2fa1-istio-envoy\") pod \"7088cc53-0ec6-43be-9669-075ca0ad2fa1\" (UID: \"7088cc53-0ec6-43be-9669-075ca0ad2fa1\") "
Oct 30 13:20:27.963901 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:27.963766    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/78fecfa2-495c-4690-bd0e-08e0a4213ade-credential-socket" pod "78fecfa2-495c-4690-bd0e-08e0a4213ade" (UID: "78fecfa2-495c-4690-bd0e-08e0a4213ade"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:20:27.963901 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:27.963777    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7088cc53-0ec6-43be-9669-075ca0ad2fa1-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7088cc53-0ec6-43be-9669-075ca0ad2fa1-workload-socket\") pod \"7088cc53-0ec6-43be-9669-075ca0ad2fa1\" (UID: \"7088cc53-0ec6-43be-9669-075ca0ad2fa1\") "
Oct 30 13:20:27.963901 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:27.963818    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/78fecfa2-495c-4690-bd0e-08e0a4213ade-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/78fecfa2-495c-4690-bd0e-08e0a4213ade-workload-socket\") pod \"78fecfa2-495c-4690-bd0e-08e0a4213ade\" (UID: \"78fecfa2-495c-4690-bd0e-08e0a4213ade\") "
Oct 30 13:20:27.963901 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:27.963849    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7088cc53-0ec6-43be-9669-075ca0ad2fa1-istio-token\" (UniqueName: \"kubernetes.io/projected/7088cc53-0ec6-43be-9669-075ca0ad2fa1-istio-token\") pod \"7088cc53-0ec6-43be-9669-075ca0ad2fa1\" (UID: \"7088cc53-0ec6-43be-9669-075ca0ad2fa1\") "
Oct 30 13:20:27.963901 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:27.963875    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/094f15a2-03fc-4322-9bcb-c190f5a6d8c4-istio-token\" (UniqueName: \"kubernetes.io/projected/094f15a2-03fc-4322-9bcb-c190f5a6d8c4-istio-token\") pod \"094f15a2-03fc-4322-9bcb-c190f5a6d8c4\" (UID: \"094f15a2-03fc-4322-9bcb-c190f5a6d8c4\") "
Oct 30 13:20:27.964420 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:27.963943    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/094f15a2-03fc-4322-9bcb-c190f5a6d8c4-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/094f15a2-03fc-4322-9bcb-c190f5a6d8c4-istio-envoy\") pod \"094f15a2-03fc-4322-9bcb-c190f5a6d8c4\" (UID: \"094f15a2-03fc-4322-9bcb-c190f5a6d8c4\") "
Oct 30 13:20:27.964420 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:27.963975    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7088cc53-0ec6-43be-9669-075ca0ad2fa1-workload-certs" pod "7088cc53-0ec6-43be-9669-075ca0ad2fa1" (UID: "7088cc53-0ec6-43be-9669-075ca0ad2fa1"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:20:27.964420 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:27.963979    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/78fecfa2-495c-4690-bd0e-08e0a4213ade-kube-api-access-txfhd\" (UniqueName: \"kubernetes.io/projected/78fecfa2-495c-4690-bd0e-08e0a4213ade-kube-api-access-txfhd\") pod \"78fecfa2-495c-4690-bd0e-08e0a4213ade\" (UID: \"78fecfa2-495c-4690-bd0e-08e0a4213ade\") "
Oct 30 13:20:27.964420 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:27.964041    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/78fecfa2-495c-4690-bd0e-08e0a4213ade-istio-token\" (UniqueName: \"kubernetes.io/projected/78fecfa2-495c-4690-bd0e-08e0a4213ade-istio-token\") pod \"78fecfa2-495c-4690-bd0e-08e0a4213ade\" (UID: \"78fecfa2-495c-4690-bd0e-08e0a4213ade\") "
Oct 30 13:20:27.964420 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:27.964106    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/78fecfa2-495c-4690-bd0e-08e0a4213ade-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/78fecfa2-495c-4690-bd0e-08e0a4213ade-istio-envoy\") pod \"78fecfa2-495c-4690-bd0e-08e0a4213ade\" (UID: \"78fecfa2-495c-4690-bd0e-08e0a4213ade\") "
Oct 30 13:20:27.964420 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:27.964148    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/78fecfa2-495c-4690-bd0e-08e0a4213ade-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/78fecfa2-495c-4690-bd0e-08e0a4213ade-istio-data\") pod \"78fecfa2-495c-4690-bd0e-08e0a4213ade\" (UID: \"78fecfa2-495c-4690-bd0e-08e0a4213ade\") "
Oct 30 13:20:27.964420 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:27.964178    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/7088cc53-0ec6-43be-9669-075ca0ad2fa1-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7088cc53-0ec6-43be-9669-075ca0ad2fa1-istiod-ca-cert\") pod \"7088cc53-0ec6-43be-9669-075ca0ad2fa1\" (UID: \"7088cc53-0ec6-43be-9669-075ca0ad2fa1\") "
Oct 30 13:20:27.964420 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:27.964206    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/094f15a2-03fc-4322-9bcb-c190f5a6d8c4-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/094f15a2-03fc-4322-9bcb-c190f5a6d8c4-workload-certs\") pod \"094f15a2-03fc-4322-9bcb-c190f5a6d8c4\" (UID: \"094f15a2-03fc-4322-9bcb-c190f5a6d8c4\") "
Oct 30 13:20:27.964420 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:27.964238    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/094f15a2-03fc-4322-9bcb-c190f5a6d8c4-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/094f15a2-03fc-4322-9bcb-c190f5a6d8c4-credential-socket\") pod \"094f15a2-03fc-4322-9bcb-c190f5a6d8c4\" (UID: \"094f15a2-03fc-4322-9bcb-c190f5a6d8c4\") "
Oct 30 13:20:27.964420 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:27.964309    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7088cc53-0ec6-43be-9669-075ca0ad2fa1-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7088cc53-0ec6-43be-9669-075ca0ad2fa1-credential-socket\") pod \"7088cc53-0ec6-43be-9669-075ca0ad2fa1\" (UID: \"7088cc53-0ec6-43be-9669-075ca0ad2fa1\") "
Oct 30 13:20:27.964420 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:27.964348    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/094f15a2-03fc-4322-9bcb-c190f5a6d8c4-kube-api-access-z65dz\" (UniqueName: \"kubernetes.io/projected/094f15a2-03fc-4322-9bcb-c190f5a6d8c4-kube-api-access-z65dz\") pod \"094f15a2-03fc-4322-9bcb-c190f5a6d8c4\" (UID: \"094f15a2-03fc-4322-9bcb-c190f5a6d8c4\") "
Oct 30 13:20:27.964979 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:27.964458    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/78fecfa2-495c-4690-bd0e-08e0a4213ade-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/78fecfa2-495c-4690-bd0e-08e0a4213ade-workload-certs\") pod \"78fecfa2-495c-4690-bd0e-08e0a4213ade\" (UID: \"78fecfa2-495c-4690-bd0e-08e0a4213ade\") "
Oct 30 13:20:27.964979 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:27.964486    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/7088cc53-0ec6-43be-9669-075ca0ad2fa1-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7088cc53-0ec6-43be-9669-075ca0ad2fa1-istio-podinfo\") pod \"7088cc53-0ec6-43be-9669-075ca0ad2fa1\" (UID: \"7088cc53-0ec6-43be-9669-075ca0ad2fa1\") "
Oct 30 13:20:27.964979 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:27.964493    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7088cc53-0ec6-43be-9669-075ca0ad2fa1-workload-socket" pod "7088cc53-0ec6-43be-9669-075ca0ad2fa1" (UID: "7088cc53-0ec6-43be-9669-075ca0ad2fa1"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:20:27.964979 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:27.964556    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7088cc53-0ec6-43be-9669-075ca0ad2fa1-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7088cc53-0ec6-43be-9669-075ca0ad2fa1-istio-data\") pod \"7088cc53-0ec6-43be-9669-075ca0ad2fa1\" (UID: \"7088cc53-0ec6-43be-9669-075ca0ad2fa1\") "
Oct 30 13:20:27.965190 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:27.964637    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/094f15a2-03fc-4322-9bcb-c190f5a6d8c4-workload-certs" pod "094f15a2-03fc-4322-9bcb-c190f5a6d8c4" (UID: "094f15a2-03fc-4322-9bcb-c190f5a6d8c4"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:20:27.965250 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:27.965203    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7088cc53-0ec6-43be-9669-075ca0ad2fa1-istio-data" pod "7088cc53-0ec6-43be-9669-075ca0ad2fa1" (UID: "7088cc53-0ec6-43be-9669-075ca0ad2fa1"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:20:27.965404 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:27.965384    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/094f15a2-03fc-4322-9bcb-c190f5a6d8c4-credential-socket" pod "094f15a2-03fc-4322-9bcb-c190f5a6d8c4" (UID: "094f15a2-03fc-4322-9bcb-c190f5a6d8c4"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:20:27.965464 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:27.965434    4943 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/78fecfa2-495c-4690-bd0e-08e0a4213ade-credential-socket\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:20:27.965464 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:27.965458    4943 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7088cc53-0ec6-43be-9669-075ca0ad2fa1-workload-certs\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:20:27.965580 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:27.965474    4943 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7088cc53-0ec6-43be-9669-075ca0ad2fa1-workload-socket\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:20:27.965580 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:27.965489    4943 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/094f15a2-03fc-4322-9bcb-c190f5a6d8c4-workload-certs\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:20:27.965580 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:27.965503    4943 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7088cc53-0ec6-43be-9669-075ca0ad2fa1-istio-data\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:20:27.965580 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:27.965515    4943 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/094f15a2-03fc-4322-9bcb-c190f5a6d8c4-istio-data\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:20:27.965580 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:27.965544    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7088cc53-0ec6-43be-9669-075ca0ad2fa1-credential-socket" pod "7088cc53-0ec6-43be-9669-075ca0ad2fa1" (UID: "7088cc53-0ec6-43be-9669-075ca0ad2fa1"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:20:27.965820 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:27.965681    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/78fecfa2-495c-4690-bd0e-08e0a4213ade-istio-data" pod "78fecfa2-495c-4690-bd0e-08e0a4213ade" (UID: "78fecfa2-495c-4690-bd0e-08e0a4213ade"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:20:27.966225 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:27.966199    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7088cc53-0ec6-43be-9669-075ca0ad2fa1-istiod-ca-cert" pod "7088cc53-0ec6-43be-9669-075ca0ad2fa1" (UID: "7088cc53-0ec6-43be-9669-075ca0ad2fa1"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 13:20:27.966351 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:27.964781    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/78fecfa2-495c-4690-bd0e-08e0a4213ade-istiod-ca-cert" pod "78fecfa2-495c-4690-bd0e-08e0a4213ade" (UID: "78fecfa2-495c-4690-bd0e-08e0a4213ade"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 13:20:27.966351 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:27.964788    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/094f15a2-03fc-4322-9bcb-c190f5a6d8c4-istiod-ca-cert" pod "094f15a2-03fc-4322-9bcb-c190f5a6d8c4" (UID: "094f15a2-03fc-4322-9bcb-c190f5a6d8c4"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 13:20:27.966517 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:27.966493    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/78fecfa2-495c-4690-bd0e-08e0a4213ade-workload-certs" pod "78fecfa2-495c-4690-bd0e-08e0a4213ade" (UID: "78fecfa2-495c-4690-bd0e-08e0a4213ade"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:20:27.966705 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:27.966685    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/094f15a2-03fc-4322-9bcb-c190f5a6d8c4-workload-socket" pod "094f15a2-03fc-4322-9bcb-c190f5a6d8c4" (UID: "094f15a2-03fc-4322-9bcb-c190f5a6d8c4"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:20:27.968059 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:27.967998    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/094f15a2-03fc-4322-9bcb-c190f5a6d8c4-kube-api-access-z65dz" pod "094f15a2-03fc-4322-9bcb-c190f5a6d8c4" (UID: "094f15a2-03fc-4322-9bcb-c190f5a6d8c4"). InnerVolumeSpecName "kube-api-access-z65dz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:20:27.968793 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:27.968764    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/094f15a2-03fc-4322-9bcb-c190f5a6d8c4-istio-token" pod "094f15a2-03fc-4322-9bcb-c190f5a6d8c4" (UID: "094f15a2-03fc-4322-9bcb-c190f5a6d8c4"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:20:27.970225 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:27.970196    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/78fecfa2-495c-4690-bd0e-08e0a4213ade-istio-podinfo" pod "78fecfa2-495c-4690-bd0e-08e0a4213ade" (UID: "78fecfa2-495c-4690-bd0e-08e0a4213ade"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 13:20:27.970338 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:27.970310    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/78fecfa2-495c-4690-bd0e-08e0a4213ade-istio-token" pod "78fecfa2-495c-4690-bd0e-08e0a4213ade" (UID: "78fecfa2-495c-4690-bd0e-08e0a4213ade"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:20:27.970563 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:27.970533    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/7088cc53-0ec6-43be-9669-075ca0ad2fa1-istio-podinfo" pod "7088cc53-0ec6-43be-9669-075ca0ad2fa1" (UID: "7088cc53-0ec6-43be-9669-075ca0ad2fa1"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 13:20:27.970876 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:27.970854    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/78fecfa2-495c-4690-bd0e-08e0a4213ade-workload-socket" pod "78fecfa2-495c-4690-bd0e-08e0a4213ade" (UID: "78fecfa2-495c-4690-bd0e-08e0a4213ade"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:20:27.971492 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:27.971469    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7088cc53-0ec6-43be-9669-075ca0ad2fa1-istio-token" pod "7088cc53-0ec6-43be-9669-075ca0ad2fa1" (UID: "7088cc53-0ec6-43be-9669-075ca0ad2fa1"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:20:27.971492 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:27.971467    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/094f15a2-03fc-4322-9bcb-c190f5a6d8c4-istio-podinfo" pod "094f15a2-03fc-4322-9bcb-c190f5a6d8c4" (UID: "094f15a2-03fc-4322-9bcb-c190f5a6d8c4"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 13:20:27.971659 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:27.971506    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/78fecfa2-495c-4690-bd0e-08e0a4213ade-kube-api-access-txfhd" pod "78fecfa2-495c-4690-bd0e-08e0a4213ade" (UID: "78fecfa2-495c-4690-bd0e-08e0a4213ade"). InnerVolumeSpecName "kube-api-access-txfhd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:20:27.972238 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:27.972214    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7088cc53-0ec6-43be-9669-075ca0ad2fa1-istio-envoy" pod "7088cc53-0ec6-43be-9669-075ca0ad2fa1" (UID: "7088cc53-0ec6-43be-9669-075ca0ad2fa1"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:20:27.972407 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:27.972386    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/094f15a2-03fc-4322-9bcb-c190f5a6d8c4-istio-envoy" pod "094f15a2-03fc-4322-9bcb-c190f5a6d8c4" (UID: "094f15a2-03fc-4322-9bcb-c190f5a6d8c4"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:20:27.972516 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:27.972500    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7088cc53-0ec6-43be-9669-075ca0ad2fa1-kube-api-access-m2dgf" pod "7088cc53-0ec6-43be-9669-075ca0ad2fa1" (UID: "7088cc53-0ec6-43be-9669-075ca0ad2fa1"). InnerVolumeSpecName "kube-api-access-m2dgf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:20:27.973341 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:27.973322    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/78fecfa2-495c-4690-bd0e-08e0a4213ade-istio-envoy" pod "78fecfa2-495c-4690-bd0e-08e0a4213ade" (UID: "78fecfa2-495c-4690-bd0e-08e0a4213ade"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:20:27.996693 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:27.996660    4943 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-d0a18c-778df4584c-4f9hx" podUID="93220213-580a-465b-b060-3cea89bfd247" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 13:20:28.054492 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:28.054459    4943 generic.go:356] "Generic (PLEG): container finished" podID="78fecfa2-495c-4690-bd0e-08e0a4213ade" containerID="e119e449a536de7ed6161856fa5ecbae96bccab870c3312223cac6af51b4d68d" exitCode=0
Oct 30 13:20:28.054882 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:28.054524    4943 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-zmddf"
Oct 30 13:20:28.054882 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:28.054548    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-zmddf" event={"ID":"78fecfa2-495c-4690-bd0e-08e0a4213ade","Type":"ContainerDied","Data":"e119e449a536de7ed6161856fa5ecbae96bccab870c3312223cac6af51b4d68d"}
Oct 30 13:20:28.054882 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:28.054623    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-zmddf" event={"ID":"78fecfa2-495c-4690-bd0e-08e0a4213ade","Type":"ContainerDied","Data":"90e91ee83c9eaa4621b6e398372133cf0bfcb524c148842734d17b19ead9562d"}
Oct 30 13:20:28.054882 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:28.054657    4943 scope.go:122] "RemoveContainer" containerID="e119e449a536de7ed6161856fa5ecbae96bccab870c3312223cac6af51b4d68d"
Oct 30 13:20:28.056684 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:28.056653    4943 generic.go:356] "Generic (PLEG): container finished" podID="7088cc53-0ec6-43be-9669-075ca0ad2fa1" containerID="6ada3baab87084aaeea903d02bf1eaba6e0b57952457b98756be7ec64be7cc28" exitCode=0
Oct 30 13:20:28.056952 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:28.056928    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-nbdn6" event={"ID":"7088cc53-0ec6-43be-9669-075ca0ad2fa1","Type":"ContainerDied","Data":"6ada3baab87084aaeea903d02bf1eaba6e0b57952457b98756be7ec64be7cc28"}
Oct 30 13:20:28.057091 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:28.056977    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-nbdn6" event={"ID":"7088cc53-0ec6-43be-9669-075ca0ad2fa1","Type":"ContainerDied","Data":"5bd61918f44e0db36444347a19fa0832778195afeeb717408925c3b0f7eb67c9"}
Oct 30 13:20:28.057091 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:28.057059    4943 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-nbdn6"
Oct 30 13:20:28.065744 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:28.062907    4943 generic.go:356] "Generic (PLEG): container finished" podID="094f15a2-03fc-4322-9bcb-c190f5a6d8c4" containerID="d97dcbe0e2d9b367fb9764d646726b47dc3956da80a55b1a9441ee67cbd98ca9" exitCode=0
Oct 30 13:20:28.065744 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:28.062997    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-c98s9" event={"ID":"094f15a2-03fc-4322-9bcb-c190f5a6d8c4","Type":"ContainerDied","Data":"d97dcbe0e2d9b367fb9764d646726b47dc3956da80a55b1a9441ee67cbd98ca9"}
Oct 30 13:20:28.065744 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:28.063039    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-c98s9" event={"ID":"094f15a2-03fc-4322-9bcb-c190f5a6d8c4","Type":"ContainerDied","Data":"5c5ef704252d3e48acf5b80058a383df624a126e831e62b832660f35ca97f76b"}
Oct 30 13:20:28.065744 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:28.063115    4943 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-c98s9"
Oct 30 13:20:28.066631 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:28.066586    4943 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/78fecfa2-495c-4690-bd0e-08e0a4213ade-workload-certs\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:20:28.066631 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:28.066613    4943 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7088cc53-0ec6-43be-9669-075ca0ad2fa1-istio-podinfo\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:20:28.066631 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:28.066626    4943 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/094f15a2-03fc-4322-9bcb-c190f5a6d8c4-workload-socket\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:20:28.066834 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:28.066638    4943 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/094f15a2-03fc-4322-9bcb-c190f5a6d8c4-istio-podinfo\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:20:28.066834 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:28.066652    4943 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/78fecfa2-495c-4690-bd0e-08e0a4213ade-istiod-ca-cert\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:20:28.066834 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:28.066665    4943 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/094f15a2-03fc-4322-9bcb-c190f5a6d8c4-istiod-ca-cert\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:20:28.066834 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:28.066677    4943 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/78fecfa2-495c-4690-bd0e-08e0a4213ade-istio-podinfo\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:20:28.066834 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:28.066690    4943 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m2dgf\" (UniqueName: \"kubernetes.io/projected/7088cc53-0ec6-43be-9669-075ca0ad2fa1-kube-api-access-m2dgf\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:20:28.066834 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:28.066703    4943 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7088cc53-0ec6-43be-9669-075ca0ad2fa1-istio-envoy\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:20:28.066834 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:28.066716    4943 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/78fecfa2-495c-4690-bd0e-08e0a4213ade-workload-socket\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:20:28.066834 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:28.066728    4943 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7088cc53-0ec6-43be-9669-075ca0ad2fa1-istio-token\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:20:28.066834 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:28.066741    4943 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/094f15a2-03fc-4322-9bcb-c190f5a6d8c4-istio-token\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:20:28.066834 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:28.066752    4943 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/094f15a2-03fc-4322-9bcb-c190f5a6d8c4-istio-envoy\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:20:28.066834 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:28.066766    4943 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-txfhd\" (UniqueName: \"kubernetes.io/projected/78fecfa2-495c-4690-bd0e-08e0a4213ade-kube-api-access-txfhd\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:20:28.066834 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:28.066778    4943 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/78fecfa2-495c-4690-bd0e-08e0a4213ade-istio-token\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:20:28.066834 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:28.066790    4943 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/78fecfa2-495c-4690-bd0e-08e0a4213ade-istio-envoy\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:20:28.066834 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:28.066802    4943 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/78fecfa2-495c-4690-bd0e-08e0a4213ade-istio-data\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:20:28.066834 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:28.066815    4943 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7088cc53-0ec6-43be-9669-075ca0ad2fa1-istiod-ca-cert\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:20:28.066834 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:28.066827    4943 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/094f15a2-03fc-4322-9bcb-c190f5a6d8c4-credential-socket\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:20:28.066834 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:28.066840    4943 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7088cc53-0ec6-43be-9669-075ca0ad2fa1-credential-socket\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:20:28.067537 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:28.066852    4943 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z65dz\" (UniqueName: \"kubernetes.io/projected/094f15a2-03fc-4322-9bcb-c190f5a6d8c4-kube-api-access-z65dz\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:20:28.071786 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:28.069407    4943 scope.go:122] "RemoveContainer" containerID="e119e449a536de7ed6161856fa5ecbae96bccab870c3312223cac6af51b4d68d"
Oct 30 13:20:28.074503 ip-10-0-142-84 kubenswrapper[4943]: E1030 13:20:28.074457    4943 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e119e449a536de7ed6161856fa5ecbae96bccab870c3312223cac6af51b4d68d\": container with ID starting with e119e449a536de7ed6161856fa5ecbae96bccab870c3312223cac6af51b4d68d not found: ID does not exist" containerID="e119e449a536de7ed6161856fa5ecbae96bccab870c3312223cac6af51b4d68d"
Oct 30 13:20:28.074590 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:28.074515    4943 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e119e449a536de7ed6161856fa5ecbae96bccab870c3312223cac6af51b4d68d"} err="failed to get container status \"e119e449a536de7ed6161856fa5ecbae96bccab870c3312223cac6af51b4d68d\": rpc error: code = NotFound desc = could not find container \"e119e449a536de7ed6161856fa5ecbae96bccab870c3312223cac6af51b4d68d\": container with ID starting with e119e449a536de7ed6161856fa5ecbae96bccab870c3312223cac6af51b4d68d not found: ID does not exist"
Oct 30 13:20:28.074590 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:28.074538    4943 scope.go:122] "RemoveContainer" containerID="6ada3baab87084aaeea903d02bf1eaba6e0b57952457b98756be7ec64be7cc28"
Oct 30 13:20:28.088918 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:28.088891    4943 scope.go:122] "RemoveContainer" containerID="6ada3baab87084aaeea903d02bf1eaba6e0b57952457b98756be7ec64be7cc28"
Oct 30 13:20:28.090232 ip-10-0-142-84 kubenswrapper[4943]: E1030 13:20:28.090203    4943 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6ada3baab87084aaeea903d02bf1eaba6e0b57952457b98756be7ec64be7cc28\": container with ID starting with 6ada3baab87084aaeea903d02bf1eaba6e0b57952457b98756be7ec64be7cc28 not found: ID does not exist" containerID="6ada3baab87084aaeea903d02bf1eaba6e0b57952457b98756be7ec64be7cc28"
Oct 30 13:20:28.090364 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:28.090242    4943 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6ada3baab87084aaeea903d02bf1eaba6e0b57952457b98756be7ec64be7cc28"} err="failed to get container status \"6ada3baab87084aaeea903d02bf1eaba6e0b57952457b98756be7ec64be7cc28\": rpc error: code = NotFound desc = could not find container \"6ada3baab87084aaeea903d02bf1eaba6e0b57952457b98756be7ec64be7cc28\": container with ID starting with 6ada3baab87084aaeea903d02bf1eaba6e0b57952457b98756be7ec64be7cc28 not found: ID does not exist"
Oct 30 13:20:28.090364 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:28.090283    4943 scope.go:122] "RemoveContainer" containerID="d97dcbe0e2d9b367fb9764d646726b47dc3956da80a55b1a9441ee67cbd98ca9"
Oct 30 13:20:28.095454 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:28.093706    4943 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-88f6a4/distinct-2-w4-88f6a4-kserve-7b8f7bd7dc-5bt68"
Oct 30 13:20:28.115879 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:28.115860    4943 scope.go:122] "RemoveContainer" containerID="d97dcbe0e2d9b367fb9764d646726b47dc3956da80a55b1a9441ee67cbd98ca9"
Oct 30 13:20:28.116213 ip-10-0-142-84 kubenswrapper[4943]: E1030 13:20:28.116192    4943 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d97dcbe0e2d9b367fb9764d646726b47dc3956da80a55b1a9441ee67cbd98ca9\": container with ID starting with d97dcbe0e2d9b367fb9764d646726b47dc3956da80a55b1a9441ee67cbd98ca9 not found: ID does not exist" containerID="d97dcbe0e2d9b367fb9764d646726b47dc3956da80a55b1a9441ee67cbd98ca9"
Oct 30 13:20:28.116405 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:28.116222    4943 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d97dcbe0e2d9b367fb9764d646726b47dc3956da80a55b1a9441ee67cbd98ca9"} err="failed to get container status \"d97dcbe0e2d9b367fb9764d646726b47dc3956da80a55b1a9441ee67cbd98ca9\": rpc error: code = NotFound desc = could not find container \"d97dcbe0e2d9b367fb9764d646726b47dc3956da80a55b1a9441ee67cbd98ca9\": container with ID starting with d97dcbe0e2d9b367fb9764d646726b47dc3956da80a55b1a9441ee67cbd98ca9 not found: ID does not exist"
Oct 30 13:20:28.140544 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:28.140519    4943 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-zmddf"]
Oct 30 13:20:28.144574 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:28.144555    4943 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-zmddf"]
Oct 30 13:20:28.265564 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:28.265482    4943 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-c98s9"]
Oct 30 13:20:28.269821 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:28.269795    4943 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-c98s9"]
Oct 30 13:20:28.310073 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:28.310040    4943 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-nbdn6"]
Oct 30 13:20:28.315192 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:28.315163    4943 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-nbdn6"]
Oct 30 13:20:28.351193 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:28.351160    4943 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="094f15a2-03fc-4322-9bcb-c190f5a6d8c4" path="/var/lib/kubelet/pods/094f15a2-03fc-4322-9bcb-c190f5a6d8c4/volumes"
Oct 30 13:20:28.351713 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:28.351697    4943 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0b3caf04-1a9d-44ce-8d80-3335b28a36ab" path="/var/lib/kubelet/pods/0b3caf04-1a9d-44ce-8d80-3335b28a36ab/volumes"
Oct 30 13:20:28.352119 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:28.352106    4943 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="663524af-aa09-44fe-8186-654cb9668d27" path="/var/lib/kubelet/pods/663524af-aa09-44fe-8186-654cb9668d27/volumes"
Oct 30 13:20:28.352899 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:28.352884    4943 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7088cc53-0ec6-43be-9669-075ca0ad2fa1" path="/var/lib/kubelet/pods/7088cc53-0ec6-43be-9669-075ca0ad2fa1/volumes"
Oct 30 13:20:28.353355 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:28.353341    4943 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="78fecfa2-495c-4690-bd0e-08e0a4213ade" path="/var/lib/kubelet/pods/78fecfa2-495c-4690-bd0e-08e0a4213ade/volumes"
Oct 30 13:20:28.353763 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:28.353750    4943 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="dbc82b1c-1c45-4263-b8ee-090fbecff086" path="/var/lib/kubelet/pods/dbc82b1c-1c45-4263-b8ee-090fbecff086/volumes"
Oct 30 13:20:31.341864 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:31.341831    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-hdmrm"]
Oct 30 13:20:33.048326 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:33.048287    4943 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-843da301-npccp"]
Oct 30 13:20:33.066289 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:33.064567    4943 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-843da301-npccp"]
Oct 30 13:20:34.088287 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:34.083423    4943 patch_prober.go:29] interesting pod/e2e-worker-w3-d0a18c-openshift-default-547d57db97-sfbdb container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.100:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 13:20:34.088287 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:34.083555    4943 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-sfbdb" podUID="4a720f44-f6a3-48b5-8ba4-04b3214486eb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.100:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:20:34.433354 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:34.418552    4943 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8464dcdb-97a0-4d19-a4ef-1d4e637f1738" path="/var/lib/kubelet/pods/8464dcdb-97a0-4d19-a4ef-1d4e637f1738/volumes"
Oct 30 13:20:35.985079 ip-10-0-142-84 kubenswrapper[4943]: E1030 13:20:35.985040    4943 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-knqz6" containerName="extractor"
Oct 30 13:20:37.166538 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:37.166509    4943 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-d0a18c-778df4584c-4f9hx"
Oct 30 13:20:37.189382 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:37.189358    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/93220213-580a-465b-b060-3cea89bfd247-kube-api-access-n847r\" (UniqueName: \"kubernetes.io/projected/93220213-580a-465b-b060-3cea89bfd247-kube-api-access-n847r\") pod \"93220213-580a-465b-b060-3cea89bfd247\" (UID: \"93220213-580a-465b-b060-3cea89bfd247\") "
Oct 30 13:20:37.189505 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:37.189491    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/93220213-580a-465b-b060-3cea89bfd247-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/93220213-580a-465b-b060-3cea89bfd247-plugins-config-volume\") pod \"93220213-580a-465b-b060-3cea89bfd247\" (UID: \"93220213-580a-465b-b060-3cea89bfd247\") "
Oct 30 13:20:37.189848 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:37.189820    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/93220213-580a-465b-b060-3cea89bfd247-plugins-config-volume" pod "93220213-580a-465b-b060-3cea89bfd247" (UID: "93220213-580a-465b-b060-3cea89bfd247"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 13:20:37.191544 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:37.191522    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/93220213-580a-465b-b060-3cea89bfd247-kube-api-access-n847r" pod "93220213-580a-465b-b060-3cea89bfd247" (UID: "93220213-580a-465b-b060-3cea89bfd247"). InnerVolumeSpecName "kube-api-access-n847r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:20:37.238065 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:37.238002    4943 generic.go:356] "Generic (PLEG): container finished" podID="93220213-580a-465b-b060-3cea89bfd247" containerID="c02e281bfa1ce0adc4e192e59fa3f1afa783120d764a61804c6e5e1680cdd7bf" exitCode=0
Oct 30 13:20:37.238248 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:37.238156    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-d0a18c-778df4584c-4f9hx" event={"ID":"93220213-580a-465b-b060-3cea89bfd247","Type":"ContainerDied","Data":"c02e281bfa1ce0adc4e192e59fa3f1afa783120d764a61804c6e5e1680cdd7bf"}
Oct 30 13:20:37.238248 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:37.238210    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-d0a18c-778df4584c-4f9hx" event={"ID":"93220213-580a-465b-b060-3cea89bfd247","Type":"ContainerDied","Data":"8ddcadab29de337657118a5fe3a8c771a50b6d3f84f950ecd2ed5fe6f52c198a"}
Oct 30 13:20:37.238411 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:37.238283    4943 scope.go:122] "RemoveContainer" containerID="c02e281bfa1ce0adc4e192e59fa3f1afa783120d764a61804c6e5e1680cdd7bf"
Oct 30 13:20:37.238470 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:37.238422    4943 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-d0a18c-778df4584c-4f9hx"
Oct 30 13:20:37.250340 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:37.250318    4943 scope.go:122] "RemoveContainer" containerID="c02e281bfa1ce0adc4e192e59fa3f1afa783120d764a61804c6e5e1680cdd7bf"
Oct 30 13:20:37.250653 ip-10-0-142-84 kubenswrapper[4943]: E1030 13:20:37.250633    4943 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c02e281bfa1ce0adc4e192e59fa3f1afa783120d764a61804c6e5e1680cdd7bf\": container with ID starting with c02e281bfa1ce0adc4e192e59fa3f1afa783120d764a61804c6e5e1680cdd7bf not found: ID does not exist" containerID="c02e281bfa1ce0adc4e192e59fa3f1afa783120d764a61804c6e5e1680cdd7bf"
Oct 30 13:20:37.250739 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:37.250660    4943 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c02e281bfa1ce0adc4e192e59fa3f1afa783120d764a61804c6e5e1680cdd7bf"} err="failed to get container status \"c02e281bfa1ce0adc4e192e59fa3f1afa783120d764a61804c6e5e1680cdd7bf\": rpc error: code = NotFound desc = could not find container \"c02e281bfa1ce0adc4e192e59fa3f1afa783120d764a61804c6e5e1680cdd7bf\": container with ID starting with c02e281bfa1ce0adc4e192e59fa3f1afa783120d764a61804c6e5e1680cdd7bf not found: ID does not exist"
Oct 30 13:20:37.290597 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:37.290567    4943 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/93220213-580a-465b-b060-3cea89bfd247-plugins-config-volume\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:20:37.290597 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:37.290596    4943 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n847r\" (UniqueName: \"kubernetes.io/projected/93220213-580a-465b-b060-3cea89bfd247-kube-api-access-n847r\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:20:37.304824 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:37.304795    4943 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w3-d0a18c-778df4584c-4f9hx"]
Oct 30 13:20:37.309072 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:37.309045    4943 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w3-d0a18c-778df4584c-4f9hx"]
Oct 30 13:20:37.811548 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:37.811515    4943 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w3-d0a18c-b6l49"]
Oct 30 13:20:37.863819 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:37.863771    4943 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w3-d0a18c-b6l49"]
Oct 30 13:20:38.350722 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:38.350690    4943 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="08f92795-8d3e-465d-8571-1b4f5a0133b3" path="/var/lib/kubelet/pods/08f92795-8d3e-465d-8571-1b4f5a0133b3/volumes"
Oct 30 13:20:38.351097 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:38.351085    4943 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="93220213-580a-465b-b060-3cea89bfd247" path="/var/lib/kubelet/pods/93220213-580a-465b-b060-3cea89bfd247/volumes"
Oct 30 13:20:40.902285 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:40.901759    4943 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w2-f76c09-mhrhl"]
Oct 30 13:20:40.920279 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:40.916685    4943 cpu_manager.go:414] "containerMap: removing container" podUID="663524af-aa09-44fe-8186-654cb9668d27" containerName="istio-proxy"
Oct 30 13:20:40.920279 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:40.916714    4943 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="663524af-aa09-44fe-8186-654cb9668d27" containerName="istio-proxy"
Oct 30 13:20:40.920279 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:40.916731    4943 cpu_manager.go:414] "containerMap: removing container" podUID="ab205844-0e5c-4c60-846c-6f6627a10692" containerName="revoke-keys"
Oct 30 13:20:40.920279 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:40.916740    4943 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ab205844-0e5c-4c60-846c-6f6627a10692" containerName="revoke-keys"
Oct 30 13:20:40.920279 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:40.916774    4943 cpu_manager.go:414] "containerMap: removing container" podUID="7088cc53-0ec6-43be-9669-075ca0ad2fa1" containerName="istio-proxy"
Oct 30 13:20:40.920279 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:40.916784    4943 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7088cc53-0ec6-43be-9669-075ca0ad2fa1" containerName="istio-proxy"
Oct 30 13:20:40.920279 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:40.916823    4943 cpu_manager.go:414] "containerMap: removing container" podUID="93220213-580a-465b-b060-3cea89bfd247" containerName="payload-pre-processing"
Oct 30 13:20:40.920279 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:40.916833    4943 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="93220213-580a-465b-b060-3cea89bfd247" containerName="payload-pre-processing"
Oct 30 13:20:40.920279 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:40.916846    4943 cpu_manager.go:414] "containerMap: removing container" podUID="0b3caf04-1a9d-44ce-8d80-3335b28a36ab" containerName="istio-proxy"
Oct 30 13:20:40.920279 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:40.916864    4943 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0b3caf04-1a9d-44ce-8d80-3335b28a36ab" containerName="istio-proxy"
Oct 30 13:20:40.920279 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:40.916881    4943 cpu_manager.go:414] "containerMap: removing container" podUID="63dace4c-7832-47dd-84ab-2c91483a5563" containerName="main"
Oct 30 13:20:40.920279 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:40.916890    4943 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="63dace4c-7832-47dd-84ab-2c91483a5563" containerName="main"
Oct 30 13:20:40.920279 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:40.916903    4943 cpu_manager.go:414] "containerMap: removing container" podUID="dbc82b1c-1c45-4263-b8ee-090fbecff086" containerName="istio-proxy"
Oct 30 13:20:40.920279 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:40.916911    4943 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="dbc82b1c-1c45-4263-b8ee-090fbecff086" containerName="istio-proxy"
Oct 30 13:20:40.920279 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:40.916931    4943 cpu_manager.go:414] "containerMap: removing container" podUID="78fecfa2-495c-4690-bd0e-08e0a4213ade" containerName="istio-proxy"
Oct 30 13:20:40.920279 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:40.916939    4943 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="78fecfa2-495c-4690-bd0e-08e0a4213ade" containerName="istio-proxy"
Oct 30 13:20:40.920279 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:40.916955    4943 cpu_manager.go:414] "containerMap: removing container" podUID="d65e47d2-4461-406a-9b71-bb3fac4262f0" containerName="main"
Oct 30 13:20:40.920279 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:40.916962    4943 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d65e47d2-4461-406a-9b71-bb3fac4262f0" containerName="main"
Oct 30 13:20:40.920279 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:40.916981    4943 cpu_manager.go:414] "containerMap: removing container" podUID="094f15a2-03fc-4322-9bcb-c190f5a6d8c4" containerName="istio-proxy"
Oct 30 13:20:40.920279 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:40.916989    4943 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="094f15a2-03fc-4322-9bcb-c190f5a6d8c4" containerName="istio-proxy"
Oct 30 13:20:40.920279 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:40.917133    4943 memory_manager.go:365] "RemoveStaleState removing state" podUID="663524af-aa09-44fe-8186-654cb9668d27" containerName="istio-proxy"
Oct 30 13:20:40.920279 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:40.917148    4943 memory_manager.go:365] "RemoveStaleState removing state" podUID="ab205844-0e5c-4c60-846c-6f6627a10692" containerName="revoke-keys"
Oct 30 13:20:40.920279 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:40.917162    4943 memory_manager.go:365] "RemoveStaleState removing state" podUID="7088cc53-0ec6-43be-9669-075ca0ad2fa1" containerName="istio-proxy"
Oct 30 13:20:40.920279 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:40.917173    4943 memory_manager.go:365] "RemoveStaleState removing state" podUID="93220213-580a-465b-b060-3cea89bfd247" containerName="payload-pre-processing"
Oct 30 13:20:40.920279 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:40.917183    4943 memory_manager.go:365] "RemoveStaleState removing state" podUID="0b3caf04-1a9d-44ce-8d80-3335b28a36ab" containerName="istio-proxy"
Oct 30 13:20:40.920279 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:40.917195    4943 memory_manager.go:365] "RemoveStaleState removing state" podUID="63dace4c-7832-47dd-84ab-2c91483a5563" containerName="main"
Oct 30 13:20:40.920279 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:40.917204    4943 memory_manager.go:365] "RemoveStaleState removing state" podUID="dbc82b1c-1c45-4263-b8ee-090fbecff086" containerName="istio-proxy"
Oct 30 13:20:40.920279 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:40.917216    4943 memory_manager.go:365] "RemoveStaleState removing state" podUID="78fecfa2-495c-4690-bd0e-08e0a4213ade" containerName="istio-proxy"
Oct 30 13:20:40.920279 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:40.917226    4943 memory_manager.go:365] "RemoveStaleState removing state" podUID="d65e47d2-4461-406a-9b71-bb3fac4262f0" containerName="main"
Oct 30 13:20:40.920279 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:40.917236    4943 memory_manager.go:365] "RemoveStaleState removing state" podUID="094f15a2-03fc-4322-9bcb-c190f5a6d8c4" containerName="istio-proxy"
Oct 30 13:20:40.925616 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:40.924501    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w2-f76c09-mhrhl"
Oct 30 13:20:40.937747 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:40.936291    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w2-f76c09-mhrhl"]
Oct 30 13:20:41.027305 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:41.014453    4943 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Oct 30 13:20:41.027305 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:41.015109    4943 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Oct 30 13:20:41.068278 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:41.068176    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/b3dce7a0-6891-4b62-86bd-12cb7c93f94c-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-worker-w2-f76c09-mhrhl\" (UID: \"b3dce7a0-6891-4b62-86bd-12cb7c93f94c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w2-f76c09-mhrhl"
Oct 30 13:20:41.068461 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:41.068306    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/b3dce7a0-6891-4b62-86bd-12cb7c93f94c-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-worker-w2-f76c09-mhrhl\" (UID: \"b3dce7a0-6891-4b62-86bd-12cb7c93f94c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w2-f76c09-mhrhl"
Oct 30 13:20:41.078293 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:41.068541    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k7qfm\" (UniqueName: \"kubernetes.io/projected/b3dce7a0-6891-4b62-86bd-12cb7c93f94c-kube-api-access-k7qfm\") pod \"maas-api-revoke-keys-e2e-worker-w2-f76c09-mhrhl\" (UID: \"b3dce7a0-6891-4b62-86bd-12cb7c93f94c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w2-f76c09-mhrhl"
Oct 30 13:20:41.104574 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:41.104414    4943 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 13:20:41.104574 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:41.104546    4943 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-p9z57\""
Oct 30 13:20:41.265205 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:41.265168    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k7qfm\" (UniqueName: \"kubernetes.io/projected/b3dce7a0-6891-4b62-86bd-12cb7c93f94c-kube-api-access-k7qfm\") pod \"maas-api-revoke-keys-e2e-worker-w2-f76c09-mhrhl\" (UID: \"b3dce7a0-6891-4b62-86bd-12cb7c93f94c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w2-f76c09-mhrhl"
Oct 30 13:20:41.274627 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:41.274592    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/b3dce7a0-6891-4b62-86bd-12cb7c93f94c-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-worker-w2-f76c09-mhrhl\" (UID: \"b3dce7a0-6891-4b62-86bd-12cb7c93f94c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w2-f76c09-mhrhl"
Oct 30 13:20:41.274877 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:41.274854    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/b3dce7a0-6891-4b62-86bd-12cb7c93f94c-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-worker-w2-f76c09-mhrhl\" (UID: \"b3dce7a0-6891-4b62-86bd-12cb7c93f94c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w2-f76c09-mhrhl"
Oct 30 13:20:41.340292 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:41.315889    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/b3dce7a0-6891-4b62-86bd-12cb7c93f94c-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-worker-w2-f76c09-mhrhl\" (UID: \"b3dce7a0-6891-4b62-86bd-12cb7c93f94c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w2-f76c09-mhrhl"
Oct 30 13:20:41.382284 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:41.380962    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k7qfm\" (UniqueName: \"kubernetes.io/projected/b3dce7a0-6891-4b62-86bd-12cb7c93f94c-kube-api-access-k7qfm\") pod \"maas-api-revoke-keys-e2e-worker-w2-f76c09-mhrhl\" (UID: \"b3dce7a0-6891-4b62-86bd-12cb7c93f94c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w2-f76c09-mhrhl"
Oct 30 13:20:41.445339 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:41.438318    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/b3dce7a0-6891-4b62-86bd-12cb7c93f94c-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-worker-w2-f76c09-mhrhl\" (UID: \"b3dce7a0-6891-4b62-86bd-12cb7c93f94c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w2-f76c09-mhrhl"
Oct 30 13:20:41.699604 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:41.699560    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w2-f76c09-mhrhl"
Oct 30 13:20:42.045541 ip-10-0-142-84 kubenswrapper[4943]: W1030 13:20:42.045497    4943 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb3dce7a0_6891_4b62_86bd_12cb7c93f94c.slice/crio-dd5b2cd4fcf330e33229d5e93e19ab65e88af8f9742611d1fbd810dfce85a2f3 WatchSource:0}: Error finding container dd5b2cd4fcf330e33229d5e93e19ab65e88af8f9742611d1fbd810dfce85a2f3: Status 404 returned error can't find the container with id dd5b2cd4fcf330e33229d5e93e19ab65e88af8f9742611d1fbd810dfce85a2f3
Oct 30 13:20:42.046280 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:42.046240    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w2-f76c09-mhrhl"]
Oct 30 13:20:42.423397 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:42.423358    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w2-f76c09-mhrhl" event={"ID":"b3dce7a0-6891-4b62-86bd-12cb7c93f94c","Type":"ContainerStarted","Data":"d9af001b505d4c3b3668f2b8f6c768d161acdb871db59023a62fb8d8e0f20892"}
Oct 30 13:20:42.423540 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:42.423414    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w2-f76c09-mhrhl" event={"ID":"b3dce7a0-6891-4b62-86bd-12cb7c93f94c","Type":"ContainerStarted","Data":"dd5b2cd4fcf330e33229d5e93e19ab65e88af8f9742611d1fbd810dfce85a2f3"}
Oct 30 13:20:42.485571 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:42.485524    4943 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w2-f76c09-mhrhl" podStartSLOduration=2.485510146 podStartE2EDuration="2.485510146s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:20:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:20:42.468753701 +0000 UTC m=+1418.256757431" watchObservedRunningTime="2026-10-30 13:20:42.485510146 +0000 UTC m=+1418.273513879"
Oct 30 13:20:43.433167 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:43.433133    4943 generic.go:356] "Generic (PLEG): container finished" podID="b3dce7a0-6891-4b62-86bd-12cb7c93f94c" containerID="d9af001b505d4c3b3668f2b8f6c768d161acdb871db59023a62fb8d8e0f20892" exitCode=0
Oct 30 13:20:43.433625 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:43.433215    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w2-f76c09-mhrhl" event={"ID":"b3dce7a0-6891-4b62-86bd-12cb7c93f94c","Type":"ContainerDied","Data":"d9af001b505d4c3b3668f2b8f6c768d161acdb871db59023a62fb8d8e0f20892"}
Oct 30 13:20:44.067644 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:44.067610    4943 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-1d7a6b59-lxwjg"]
Oct 30 13:20:44.083430 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:44.083393    4943 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-1d7a6b59-lxwjg"]
Oct 30 13:20:44.402739 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:44.402667    4943 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="14a99554-9231-4607-b677-692c441093b0" path="/var/lib/kubelet/pods/14a99554-9231-4607-b677-692c441093b0/volumes"
Oct 30 13:20:44.595469 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:44.595443    4943 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w2-f76c09-mhrhl"
Oct 30 13:20:44.724321 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:44.724291    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b3dce7a0-6891-4b62-86bd-12cb7c93f94c-kube-api-access-k7qfm\" (UniqueName: \"kubernetes.io/projected/b3dce7a0-6891-4b62-86bd-12cb7c93f94c-kube-api-access-k7qfm\") pod \"b3dce7a0-6891-4b62-86bd-12cb7c93f94c\" (UID: \"b3dce7a0-6891-4b62-86bd-12cb7c93f94c\") "
Oct 30 13:20:44.724490 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:44.724335    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b3dce7a0-6891-4b62-86bd-12cb7c93f94c-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/b3dce7a0-6891-4b62-86bd-12cb7c93f94c-maas-api-openshift-ca\") pod \"b3dce7a0-6891-4b62-86bd-12cb7c93f94c\" (UID: \"b3dce7a0-6891-4b62-86bd-12cb7c93f94c\") "
Oct 30 13:20:44.724547 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:44.724510    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/b3dce7a0-6891-4b62-86bd-12cb7c93f94c-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/b3dce7a0-6891-4b62-86bd-12cb7c93f94c-maas-api-xks-ca\") pod \"b3dce7a0-6891-4b62-86bd-12cb7c93f94c\" (UID: \"b3dce7a0-6891-4b62-86bd-12cb7c93f94c\") "
Oct 30 13:20:44.724773 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:44.724749    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b3dce7a0-6891-4b62-86bd-12cb7c93f94c-maas-api-openshift-ca" pod "b3dce7a0-6891-4b62-86bd-12cb7c93f94c" (UID: "b3dce7a0-6891-4b62-86bd-12cb7c93f94c"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 13:20:44.724986 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:44.724965    4943 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/b3dce7a0-6891-4b62-86bd-12cb7c93f94c-maas-api-openshift-ca\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:20:44.726644 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:44.726613    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b3dce7a0-6891-4b62-86bd-12cb7c93f94c-maas-api-xks-ca" pod "b3dce7a0-6891-4b62-86bd-12cb7c93f94c" (UID: "b3dce7a0-6891-4b62-86bd-12cb7c93f94c"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 13:20:44.726729 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:44.726671    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b3dce7a0-6891-4b62-86bd-12cb7c93f94c-kube-api-access-k7qfm" pod "b3dce7a0-6891-4b62-86bd-12cb7c93f94c" (UID: "b3dce7a0-6891-4b62-86bd-12cb7c93f94c"). InnerVolumeSpecName "kube-api-access-k7qfm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:20:44.825584 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:44.825556    4943 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/b3dce7a0-6891-4b62-86bd-12cb7c93f94c-maas-api-xks-ca\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:20:44.825584 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:44.825583    4943 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k7qfm\" (UniqueName: \"kubernetes.io/projected/b3dce7a0-6891-4b62-86bd-12cb7c93f94c-kube-api-access-k7qfm\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:20:45.449917 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:45.449889    4943 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w2-f76c09-mhrhl"
Oct 30 13:20:45.450098 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:45.449932    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w2-f76c09-mhrhl" event={"ID":"b3dce7a0-6891-4b62-86bd-12cb7c93f94c","Type":"ContainerDied","Data":"dd5b2cd4fcf330e33229d5e93e19ab65e88af8f9742611d1fbd810dfce85a2f3"}
Oct 30 13:20:45.450098 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:45.449982    4943 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="dd5b2cd4fcf330e33229d5e93e19ab65e88af8f9742611d1fbd810dfce85a2f3"
Oct 30 13:20:45.686090 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:45.686054    4943 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w2-f76c09/distinct-2-w2-f76c09-kserve-78446dcc4c-7ljvf"]
Oct 30 13:20:47.079108 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:47.079068    4943 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-66cf4d45-xxxx2"]
Oct 30 13:20:47.090496 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:47.090464    4943 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-66cf4d45-xxxx2"]
Oct 30 13:20:47.268301 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:47.263103    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-hdmrm"]
Oct 30 13:20:48.380028 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:48.379969    4943 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="606ec1b4-0953-4cfc-a7e9-ea7832a39f11" path="/var/lib/kubelet/pods/606ec1b4-0953-4cfc-a7e9-ea7832a39f11/volumes"
Oct 30 13:20:48.796397 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:48.796362    4943 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-9ndzw"]
Oct 30 13:20:48.796612 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:48.796587    4943 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-9ndzw" podUID="70c62fc1-3452-47f0-8d1f-5495e535cd95" containerName="istio-proxy" containerID="cri-o://eead86c384c8ed3785b2c45cc493e7003a79c6c27a92ccc315f6cbca01aa5676" gracePeriod=30
Oct 30 13:20:48.810494 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:48.810464    4943 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-t7zm7"]
Oct 30 13:20:48.810663 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:48.810643    4943 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-t7zm7" podUID="0ee4232a-d7fe-4aa9-963c-4bac5daa86c3" containerName="istio-proxy" containerID="cri-o://b08612a8773ce4d0dae9d62e24dd8ab0b64f53425ac633e21d4c06ed876f2696" gracePeriod=30
Oct 30 13:20:48.831856 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:48.831830    4943 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-2zhxj"]
Oct 30 13:20:48.832030 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:48.832011    4943 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-2zhxj" podUID="cf47ace6-4db7-4896-98a4-496dab0e722f" containerName="istio-proxy" containerID="cri-o://479a3b20dd15fe9271601fddfd828062fcff28532944df485c2ebee2756c0d76" gracePeriod=30
Oct 30 13:20:48.840585 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:48.840559    4943 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-sfbdb"]
Oct 30 13:20:48.840765 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:48.840744    4943 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-sfbdb" podUID="4a720f44-f6a3-48b5-8ba4-04b3214486eb" containerName="istio-proxy" containerID="cri-o://a154f39e24a8c3c65170b5213a6d298fbfbe40344cc4164fa7794afbf59fc265" gracePeriod=30
Oct 30 13:20:50.470668 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:50.470632    4943 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-88f6a4-7tfvz"]
Oct 30 13:20:50.471207 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:50.471192    4943 cpu_manager.go:414] "containerMap: removing container" podUID="b3dce7a0-6891-4b62-86bd-12cb7c93f94c" containerName="revoke-keys"
Oct 30 13:20:50.471299 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:50.471207    4943 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b3dce7a0-6891-4b62-86bd-12cb7c93f94c" containerName="revoke-keys"
Oct 30 13:20:50.471345 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:50.471302    4943 memory_manager.go:365] "RemoveStaleState removing state" podUID="b3dce7a0-6891-4b62-86bd-12cb7c93f94c" containerName="revoke-keys"
Oct 30 13:20:50.476326 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:50.476305    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-88f6a4-7tfvz"
Oct 30 13:20:50.503990 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:50.503971    4943 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Oct 30 13:20:50.516862 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:50.516832    4943 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 13:20:50.518687 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:50.518669    4943 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Oct 30 13:20:50.519148 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:50.519127    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/5b325a80-09c9-4350-8e01-f50cfab3e2a7-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-worker-w4-88f6a4-7tfvz\" (UID: \"5b325a80-09c9-4350-8e01-f50cfab3e2a7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-88f6a4-7tfvz"
Oct 30 13:20:50.519182 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:50.519161    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9klv7\" (UniqueName: \"kubernetes.io/projected/5b325a80-09c9-4350-8e01-f50cfab3e2a7-kube-api-access-9klv7\") pod \"maas-api-revoke-keys-e2e-worker-w4-88f6a4-7tfvz\" (UID: \"5b325a80-09c9-4350-8e01-f50cfab3e2a7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-88f6a4-7tfvz"
Oct 30 13:20:50.519245 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:50.519232    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/5b325a80-09c9-4350-8e01-f50cfab3e2a7-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-worker-w4-88f6a4-7tfvz\" (UID: \"5b325a80-09c9-4350-8e01-f50cfab3e2a7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-88f6a4-7tfvz"
Oct 30 13:20:50.522122 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:50.522105    4943 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-p9z57\""
Oct 30 13:20:50.564900 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:50.564873    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-88f6a4-7tfvz"]
Oct 30 13:20:50.620746 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:50.620720    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/5b325a80-09c9-4350-8e01-f50cfab3e2a7-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-worker-w4-88f6a4-7tfvz\" (UID: \"5b325a80-09c9-4350-8e01-f50cfab3e2a7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-88f6a4-7tfvz"
Oct 30 13:20:50.620851 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:50.620754    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9klv7\" (UniqueName: \"kubernetes.io/projected/5b325a80-09c9-4350-8e01-f50cfab3e2a7-kube-api-access-9klv7\") pod \"maas-api-revoke-keys-e2e-worker-w4-88f6a4-7tfvz\" (UID: \"5b325a80-09c9-4350-8e01-f50cfab3e2a7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-88f6a4-7tfvz"
Oct 30 13:20:50.620851 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:50.620805    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/5b325a80-09c9-4350-8e01-f50cfab3e2a7-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-worker-w4-88f6a4-7tfvz\" (UID: \"5b325a80-09c9-4350-8e01-f50cfab3e2a7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-88f6a4-7tfvz"
Oct 30 13:20:50.621368 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:50.621348    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/5b325a80-09c9-4350-8e01-f50cfab3e2a7-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-worker-w4-88f6a4-7tfvz\" (UID: \"5b325a80-09c9-4350-8e01-f50cfab3e2a7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-88f6a4-7tfvz"
Oct 30 13:20:50.623252 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:50.623231    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/5b325a80-09c9-4350-8e01-f50cfab3e2a7-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-worker-w4-88f6a4-7tfvz\" (UID: \"5b325a80-09c9-4350-8e01-f50cfab3e2a7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-88f6a4-7tfvz"
Oct 30 13:20:50.662163 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:50.662143    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9klv7\" (UniqueName: \"kubernetes.io/projected/5b325a80-09c9-4350-8e01-f50cfab3e2a7-kube-api-access-9klv7\") pod \"maas-api-revoke-keys-e2e-worker-w4-88f6a4-7tfvz\" (UID: \"5b325a80-09c9-4350-8e01-f50cfab3e2a7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-88f6a4-7tfvz"
Oct 30 13:20:50.790312 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:50.790236    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-88f6a4-7tfvz"
Oct 30 13:20:51.070604 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:51.070528    4943 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-49abdeb7-g7kx7"]
Oct 30 13:20:51.083594 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:51.083564    4943 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-49abdeb7-g7kx7"]
Oct 30 13:20:51.149822 ip-10-0-142-84 kubenswrapper[4943]: W1030 13:20:51.149779    4943 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5b325a80_09c9_4350_8e01_f50cfab3e2a7.slice/crio-9b08893b3a4475f7ab83be7232f5e522b7a3588b69bb8187741c920ad281829c WatchSource:0}: Error finding container 9b08893b3a4475f7ab83be7232f5e522b7a3588b69bb8187741c920ad281829c: Status 404 returned error can't find the container with id 9b08893b3a4475f7ab83be7232f5e522b7a3588b69bb8187741c920ad281829c
Oct 30 13:20:51.151343 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:51.151303    4943 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w2-f76c09-588d998d5b-z94mv"]
Oct 30 13:20:51.167981 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:51.167945    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-88f6a4-7tfvz"]
Oct 30 13:20:51.225215 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:51.225186    4943 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w2-f76c09-5bdd4bf999-gbp8q"]
Oct 30 13:20:51.529160 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:51.529120    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-88f6a4-7tfvz" event={"ID":"5b325a80-09c9-4350-8e01-f50cfab3e2a7","Type":"ContainerStarted","Data":"9b08893b3a4475f7ab83be7232f5e522b7a3588b69bb8187741c920ad281829c"}
Oct 30 13:20:52.351192 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:52.351157    4943 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ef06d03d-a788-4c5e-919c-c67521941405" path="/var/lib/kubelet/pods/ef06d03d-a788-4c5e-919c-c67521941405/volumes"
Oct 30 13:20:52.538157 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:52.538124    4943 generic.go:356] "Generic (PLEG): container finished" podID="5b325a80-09c9-4350-8e01-f50cfab3e2a7" containerID="60b9bfbc44521f71748b1ae34d58bd035f476f43db06394c522abb9d2bf937c0" exitCode=0
Oct 30 13:20:52.538585 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:52.538208    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-88f6a4-7tfvz" event={"ID":"5b325a80-09c9-4350-8e01-f50cfab3e2a7","Type":"ContainerDied","Data":"60b9bfbc44521f71748b1ae34d58bd035f476f43db06394c522abb9d2bf937c0"}
Oct 30 13:20:52.903892 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:52.903816    4943 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-586tt"]
Oct 30 13:20:52.907790 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:52.907774    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-586tt"
Oct 30 13:20:52.952056 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:52.952017    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-586tt"]
Oct 30 13:20:52.966503 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:52.966468    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9c5b6db1-55db-447c-a3a9-2e24a9dc9dc6-istio-data\") pod \"e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-586tt\" (UID: \"9c5b6db1-55db-447c-a3a9-2e24a9dc9dc6\") " pod="openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-586tt"
Oct 30 13:20:52.966668 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:52.966509    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-92npq\" (UniqueName: \"kubernetes.io/projected/9c5b6db1-55db-447c-a3a9-2e24a9dc9dc6-kube-api-access-92npq\") pod \"e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-586tt\" (UID: \"9c5b6db1-55db-447c-a3a9-2e24a9dc9dc6\") " pod="openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-586tt"
Oct 30 13:20:52.966668 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:52.966633    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9c5b6db1-55db-447c-a3a9-2e24a9dc9dc6-istio-envoy\") pod \"e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-586tt\" (UID: \"9c5b6db1-55db-447c-a3a9-2e24a9dc9dc6\") " pod="openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-586tt"
Oct 30 13:20:52.966668 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:52.966659    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9c5b6db1-55db-447c-a3a9-2e24a9dc9dc6-istiod-ca-cert\") pod \"e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-586tt\" (UID: \"9c5b6db1-55db-447c-a3a9-2e24a9dc9dc6\") " pod="openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-586tt"
Oct 30 13:20:52.966854 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:52.966698    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9c5b6db1-55db-447c-a3a9-2e24a9dc9dc6-istio-podinfo\") pod \"e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-586tt\" (UID: \"9c5b6db1-55db-447c-a3a9-2e24a9dc9dc6\") " pod="openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-586tt"
Oct 30 13:20:52.966854 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:52.966732    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9c5b6db1-55db-447c-a3a9-2e24a9dc9dc6-workload-socket\") pod \"e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-586tt\" (UID: \"9c5b6db1-55db-447c-a3a9-2e24a9dc9dc6\") " pod="openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-586tt"
Oct 30 13:20:52.966854 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:52.966748    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9c5b6db1-55db-447c-a3a9-2e24a9dc9dc6-credential-socket\") pod \"e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-586tt\" (UID: \"9c5b6db1-55db-447c-a3a9-2e24a9dc9dc6\") " pod="openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-586tt"
Oct 30 13:20:52.966854 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:52.966793    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9c5b6db1-55db-447c-a3a9-2e24a9dc9dc6-workload-certs\") pod \"e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-586tt\" (UID: \"9c5b6db1-55db-447c-a3a9-2e24a9dc9dc6\") " pod="openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-586tt"
Oct 30 13:20:52.966854 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:52.966812    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9c5b6db1-55db-447c-a3a9-2e24a9dc9dc6-istio-token\") pod \"e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-586tt\" (UID: \"9c5b6db1-55db-447c-a3a9-2e24a9dc9dc6\") " pod="openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-586tt"
Oct 30 13:20:53.067883 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:53.067841    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9c5b6db1-55db-447c-a3a9-2e24a9dc9dc6-istio-token\") pod \"e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-586tt\" (UID: \"9c5b6db1-55db-447c-a3a9-2e24a9dc9dc6\") " pod="openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-586tt"
Oct 30 13:20:53.068049 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:53.067932    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9c5b6db1-55db-447c-a3a9-2e24a9dc9dc6-istio-data\") pod \"e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-586tt\" (UID: \"9c5b6db1-55db-447c-a3a9-2e24a9dc9dc6\") " pod="openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-586tt"
Oct 30 13:20:53.068049 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:53.067963    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-92npq\" (UniqueName: \"kubernetes.io/projected/9c5b6db1-55db-447c-a3a9-2e24a9dc9dc6-kube-api-access-92npq\") pod \"e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-586tt\" (UID: \"9c5b6db1-55db-447c-a3a9-2e24a9dc9dc6\") " pod="openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-586tt"
Oct 30 13:20:53.068163 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:53.068077    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9c5b6db1-55db-447c-a3a9-2e24a9dc9dc6-istio-envoy\") pod \"e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-586tt\" (UID: \"9c5b6db1-55db-447c-a3a9-2e24a9dc9dc6\") " pod="openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-586tt"
Oct 30 13:20:53.068285 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:53.068227    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9c5b6db1-55db-447c-a3a9-2e24a9dc9dc6-istiod-ca-cert\") pod \"e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-586tt\" (UID: \"9c5b6db1-55db-447c-a3a9-2e24a9dc9dc6\") " pod="openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-586tt"
Oct 30 13:20:53.068376 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:53.068330    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9c5b6db1-55db-447c-a3a9-2e24a9dc9dc6-istio-podinfo\") pod \"e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-586tt\" (UID: \"9c5b6db1-55db-447c-a3a9-2e24a9dc9dc6\") " pod="openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-586tt"
Oct 30 13:20:53.068444 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:53.068390    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9c5b6db1-55db-447c-a3a9-2e24a9dc9dc6-workload-socket\") pod \"e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-586tt\" (UID: \"9c5b6db1-55db-447c-a3a9-2e24a9dc9dc6\") " pod="openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-586tt"
Oct 30 13:20:53.068444 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:53.068418    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9c5b6db1-55db-447c-a3a9-2e24a9dc9dc6-credential-socket\") pod \"e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-586tt\" (UID: \"9c5b6db1-55db-447c-a3a9-2e24a9dc9dc6\") " pod="openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-586tt"
Oct 30 13:20:53.068444 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:53.068425    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9c5b6db1-55db-447c-a3a9-2e24a9dc9dc6-istio-data\") pod \"e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-586tt\" (UID: \"9c5b6db1-55db-447c-a3a9-2e24a9dc9dc6\") " pod="openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-586tt"
Oct 30 13:20:53.068613 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:53.068502    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9c5b6db1-55db-447c-a3a9-2e24a9dc9dc6-workload-certs\") pod \"e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-586tt\" (UID: \"9c5b6db1-55db-447c-a3a9-2e24a9dc9dc6\") " pod="openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-586tt"
Oct 30 13:20:53.069001 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:53.068687    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9c5b6db1-55db-447c-a3a9-2e24a9dc9dc6-credential-socket\") pod \"e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-586tt\" (UID: \"9c5b6db1-55db-447c-a3a9-2e24a9dc9dc6\") " pod="openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-586tt"
Oct 30 13:20:53.069001 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:53.068842    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9c5b6db1-55db-447c-a3a9-2e24a9dc9dc6-workload-certs\") pod \"e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-586tt\" (UID: \"9c5b6db1-55db-447c-a3a9-2e24a9dc9dc6\") " pod="openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-586tt"
Oct 30 13:20:53.069001 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:53.068959    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9c5b6db1-55db-447c-a3a9-2e24a9dc9dc6-workload-socket\") pod \"e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-586tt\" (UID: \"9c5b6db1-55db-447c-a3a9-2e24a9dc9dc6\") " pod="openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-586tt"
Oct 30 13:20:53.069203 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:53.069095    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9c5b6db1-55db-447c-a3a9-2e24a9dc9dc6-istiod-ca-cert\") pod \"e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-586tt\" (UID: \"9c5b6db1-55db-447c-a3a9-2e24a9dc9dc6\") " pod="openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-586tt"
Oct 30 13:20:53.071003 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:53.070974    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9c5b6db1-55db-447c-a3a9-2e24a9dc9dc6-istio-envoy\") pod \"e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-586tt\" (UID: \"9c5b6db1-55db-447c-a3a9-2e24a9dc9dc6\") " pod="openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-586tt"
Oct 30 13:20:53.071141 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:53.071124    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9c5b6db1-55db-447c-a3a9-2e24a9dc9dc6-istio-podinfo\") pod \"e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-586tt\" (UID: \"9c5b6db1-55db-447c-a3a9-2e24a9dc9dc6\") " pod="openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-586tt"
Oct 30 13:20:53.092968 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:53.092948    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9c5b6db1-55db-447c-a3a9-2e24a9dc9dc6-istio-token\") pod \"e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-586tt\" (UID: \"9c5b6db1-55db-447c-a3a9-2e24a9dc9dc6\") " pod="openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-586tt"
Oct 30 13:20:53.093189 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:53.093171    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-92npq\" (UniqueName: \"kubernetes.io/projected/9c5b6db1-55db-447c-a3a9-2e24a9dc9dc6-kube-api-access-92npq\") pod \"e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-586tt\" (UID: \"9c5b6db1-55db-447c-a3a9-2e24a9dc9dc6\") " pod="openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-586tt"
Oct 30 13:20:53.223826 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:53.223800    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-586tt"
Oct 30 13:20:53.571573 ip-10-0-142-84 kubenswrapper[4943]: W1030 13:20:53.571535    4943 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9c5b6db1_55db_447c_a3a9_2e24a9dc9dc6.slice/crio-24993988e17052f2727f3f8b1cafabeee52d1c253548cb432921cbd5915d4ead WatchSource:0}: Error finding container 24993988e17052f2727f3f8b1cafabeee52d1c253548cb432921cbd5915d4ead: Status 404 returned error can't find the container with id 24993988e17052f2727f3f8b1cafabeee52d1c253548cb432921cbd5915d4ead
Oct 30 13:20:53.574619 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:53.574578    4943 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 13:20:53.574732 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:53.574662    4943 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 13:20:53.574732 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:53.574699    4943 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 13:20:53.575122 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:53.575094    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-586tt"]
Oct 30 13:20:53.736483 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:53.736455    4943 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-88f6a4-7tfvz"
Oct 30 13:20:53.774596 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:53.774564    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5b325a80-09c9-4350-8e01-f50cfab3e2a7-kube-api-access-9klv7\" (UniqueName: \"kubernetes.io/projected/5b325a80-09c9-4350-8e01-f50cfab3e2a7-kube-api-access-9klv7\") pod \"5b325a80-09c9-4350-8e01-f50cfab3e2a7\" (UID: \"5b325a80-09c9-4350-8e01-f50cfab3e2a7\") "
Oct 30 13:20:53.774788 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:53.774634    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5b325a80-09c9-4350-8e01-f50cfab3e2a7-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/5b325a80-09c9-4350-8e01-f50cfab3e2a7-maas-api-openshift-ca\") pod \"5b325a80-09c9-4350-8e01-f50cfab3e2a7\" (UID: \"5b325a80-09c9-4350-8e01-f50cfab3e2a7\") "
Oct 30 13:20:53.774788 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:53.774718    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/5b325a80-09c9-4350-8e01-f50cfab3e2a7-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/5b325a80-09c9-4350-8e01-f50cfab3e2a7-maas-api-xks-ca\") pod \"5b325a80-09c9-4350-8e01-f50cfab3e2a7\" (UID: \"5b325a80-09c9-4350-8e01-f50cfab3e2a7\") "
Oct 30 13:20:53.775220 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:53.775189    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5b325a80-09c9-4350-8e01-f50cfab3e2a7-maas-api-openshift-ca" pod "5b325a80-09c9-4350-8e01-f50cfab3e2a7" (UID: "5b325a80-09c9-4350-8e01-f50cfab3e2a7"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 13:20:53.777200 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:53.777176    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5b325a80-09c9-4350-8e01-f50cfab3e2a7-kube-api-access-9klv7" pod "5b325a80-09c9-4350-8e01-f50cfab3e2a7" (UID: "5b325a80-09c9-4350-8e01-f50cfab3e2a7"). InnerVolumeSpecName "kube-api-access-9klv7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:20:53.777404 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:53.777382    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5b325a80-09c9-4350-8e01-f50cfab3e2a7-maas-api-xks-ca" pod "5b325a80-09c9-4350-8e01-f50cfab3e2a7" (UID: "5b325a80-09c9-4350-8e01-f50cfab3e2a7"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 13:20:53.876384 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:53.876359    4943 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/5b325a80-09c9-4350-8e01-f50cfab3e2a7-maas-api-xks-ca\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:20:53.876490 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:53.876386    4943 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9klv7\" (UniqueName: \"kubernetes.io/projected/5b325a80-09c9-4350-8e01-f50cfab3e2a7-kube-api-access-9klv7\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:20:53.876490 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:53.876396    4943 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/5b325a80-09c9-4350-8e01-f50cfab3e2a7-maas-api-openshift-ca\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:20:54.137762 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:54.137735    4943 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-2zhxj"
Oct 30 13:20:54.187295 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:54.187245    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/cf47ace6-4db7-4896-98a4-496dab0e722f-istio-token\" (UniqueName: \"kubernetes.io/projected/cf47ace6-4db7-4896-98a4-496dab0e722f-istio-token\") pod \"cf47ace6-4db7-4896-98a4-496dab0e722f\" (UID: \"cf47ace6-4db7-4896-98a4-496dab0e722f\") "
Oct 30 13:20:54.187528 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:54.187361    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cf47ace6-4db7-4896-98a4-496dab0e722f-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cf47ace6-4db7-4896-98a4-496dab0e722f-workload-certs\") pod \"cf47ace6-4db7-4896-98a4-496dab0e722f\" (UID: \"cf47ace6-4db7-4896-98a4-496dab0e722f\") "
Oct 30 13:20:54.187528 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:54.187392    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/cf47ace6-4db7-4896-98a4-496dab0e722f-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cf47ace6-4db7-4896-98a4-496dab0e722f-istio-podinfo\") pod \"cf47ace6-4db7-4896-98a4-496dab0e722f\" (UID: \"cf47ace6-4db7-4896-98a4-496dab0e722f\") "
Oct 30 13:20:54.187528 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:54.187480    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cf47ace6-4db7-4896-98a4-496dab0e722f-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf47ace6-4db7-4896-98a4-496dab0e722f-credential-socket\") pod \"cf47ace6-4db7-4896-98a4-496dab0e722f\" (UID: \"cf47ace6-4db7-4896-98a4-496dab0e722f\") "
Oct 30 13:20:54.187729 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:54.187541    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cf47ace6-4db7-4896-98a4-496dab0e722f-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf47ace6-4db7-4896-98a4-496dab0e722f-workload-socket\") pod \"cf47ace6-4db7-4896-98a4-496dab0e722f\" (UID: \"cf47ace6-4db7-4896-98a4-496dab0e722f\") "
Oct 30 13:20:54.187729 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:54.187622    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cf47ace6-4db7-4896-98a4-496dab0e722f-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cf47ace6-4db7-4896-98a4-496dab0e722f-istio-data\") pod \"cf47ace6-4db7-4896-98a4-496dab0e722f\" (UID: \"cf47ace6-4db7-4896-98a4-496dab0e722f\") "
Oct 30 13:20:54.187729 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:54.187684    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/cf47ace6-4db7-4896-98a4-496dab0e722f-kube-api-access-zp2tx\" (UniqueName: \"kubernetes.io/projected/cf47ace6-4db7-4896-98a4-496dab0e722f-kube-api-access-zp2tx\") pod \"cf47ace6-4db7-4896-98a4-496dab0e722f\" (UID: \"cf47ace6-4db7-4896-98a4-496dab0e722f\") "
Oct 30 13:20:54.187898 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:54.187764    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/cf47ace6-4db7-4896-98a4-496dab0e722f-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cf47ace6-4db7-4896-98a4-496dab0e722f-istiod-ca-cert\") pod \"cf47ace6-4db7-4896-98a4-496dab0e722f\" (UID: \"cf47ace6-4db7-4896-98a4-496dab0e722f\") "
Oct 30 13:20:54.187898 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:54.187789    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cf47ace6-4db7-4896-98a4-496dab0e722f-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cf47ace6-4db7-4896-98a4-496dab0e722f-istio-envoy\") pod \"cf47ace6-4db7-4896-98a4-496dab0e722f\" (UID: \"cf47ace6-4db7-4896-98a4-496dab0e722f\") "
Oct 30 13:20:54.189182 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:54.189042    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cf47ace6-4db7-4896-98a4-496dab0e722f-workload-certs" pod "cf47ace6-4db7-4896-98a4-496dab0e722f" (UID: "cf47ace6-4db7-4896-98a4-496dab0e722f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:20:54.202709 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:54.189223    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cf47ace6-4db7-4896-98a4-496dab0e722f-credential-socket" pod "cf47ace6-4db7-4896-98a4-496dab0e722f" (UID: "cf47ace6-4db7-4896-98a4-496dab0e722f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:20:54.202709 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:54.189417    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cf47ace6-4db7-4896-98a4-496dab0e722f-workload-socket" pod "cf47ace6-4db7-4896-98a4-496dab0e722f" (UID: "cf47ace6-4db7-4896-98a4-496dab0e722f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:20:54.202709 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:54.189550    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cf47ace6-4db7-4896-98a4-496dab0e722f-istiod-ca-cert" pod "cf47ace6-4db7-4896-98a4-496dab0e722f" (UID: "cf47ace6-4db7-4896-98a4-496dab0e722f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 13:20:54.203588 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:54.203026    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cf47ace6-4db7-4896-98a4-496dab0e722f-istio-data" pod "cf47ace6-4db7-4896-98a4-496dab0e722f" (UID: "cf47ace6-4db7-4896-98a4-496dab0e722f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:20:54.211497 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:54.211447    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cf47ace6-4db7-4896-98a4-496dab0e722f-kube-api-access-zp2tx" pod "cf47ace6-4db7-4896-98a4-496dab0e722f" (UID: "cf47ace6-4db7-4896-98a4-496dab0e722f"). InnerVolumeSpecName "kube-api-access-zp2tx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:20:54.211753 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:54.211718    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cf47ace6-4db7-4896-98a4-496dab0e722f-istio-envoy" pod "cf47ace6-4db7-4896-98a4-496dab0e722f" (UID: "cf47ace6-4db7-4896-98a4-496dab0e722f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:20:54.213963 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:54.213928    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cf47ace6-4db7-4896-98a4-496dab0e722f-istio-token" pod "cf47ace6-4db7-4896-98a4-496dab0e722f" (UID: "cf47ace6-4db7-4896-98a4-496dab0e722f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:20:54.220834 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:54.219673    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/cf47ace6-4db7-4896-98a4-496dab0e722f-istio-podinfo" pod "cf47ace6-4db7-4896-98a4-496dab0e722f" (UID: "cf47ace6-4db7-4896-98a4-496dab0e722f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 13:20:54.293564 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:54.276949    4943 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-9ndzw"
Oct 30 13:20:54.304198 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:54.304164    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/70c62fc1-3452-47f0-8d1f-5495e535cd95-kube-api-access-t87zw\" (UniqueName: \"kubernetes.io/projected/70c62fc1-3452-47f0-8d1f-5495e535cd95-kube-api-access-t87zw\") pod \"70c62fc1-3452-47f0-8d1f-5495e535cd95\" (UID: \"70c62fc1-3452-47f0-8d1f-5495e535cd95\") "
Oct 30 13:20:54.304439 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:54.304215    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/70c62fc1-3452-47f0-8d1f-5495e535cd95-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/70c62fc1-3452-47f0-8d1f-5495e535cd95-workload-socket\") pod \"70c62fc1-3452-47f0-8d1f-5495e535cd95\" (UID: \"70c62fc1-3452-47f0-8d1f-5495e535cd95\") "
Oct 30 13:20:54.304439 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:54.304284    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/70c62fc1-3452-47f0-8d1f-5495e535cd95-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/70c62fc1-3452-47f0-8d1f-5495e535cd95-istiod-ca-cert\") pod \"70c62fc1-3452-47f0-8d1f-5495e535cd95\" (UID: \"70c62fc1-3452-47f0-8d1f-5495e535cd95\") "
Oct 30 13:20:54.304439 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:54.304321    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/70c62fc1-3452-47f0-8d1f-5495e535cd95-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/70c62fc1-3452-47f0-8d1f-5495e535cd95-istio-envoy\") pod \"70c62fc1-3452-47f0-8d1f-5495e535cd95\" (UID: \"70c62fc1-3452-47f0-8d1f-5495e535cd95\") "
Oct 30 13:20:54.304439 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:54.304352    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/70c62fc1-3452-47f0-8d1f-5495e535cd95-istio-token\" (UniqueName: \"kubernetes.io/projected/70c62fc1-3452-47f0-8d1f-5495e535cd95-istio-token\") pod \"70c62fc1-3452-47f0-8d1f-5495e535cd95\" (UID: \"70c62fc1-3452-47f0-8d1f-5495e535cd95\") "
Oct 30 13:20:54.304439 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:54.304414    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/70c62fc1-3452-47f0-8d1f-5495e535cd95-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/70c62fc1-3452-47f0-8d1f-5495e535cd95-istio-podinfo\") pod \"70c62fc1-3452-47f0-8d1f-5495e535cd95\" (UID: \"70c62fc1-3452-47f0-8d1f-5495e535cd95\") "
Oct 30 13:20:54.304695 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:54.304463    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/70c62fc1-3452-47f0-8d1f-5495e535cd95-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/70c62fc1-3452-47f0-8d1f-5495e535cd95-workload-certs\") pod \"70c62fc1-3452-47f0-8d1f-5495e535cd95\" (UID: \"70c62fc1-3452-47f0-8d1f-5495e535cd95\") "
Oct 30 13:20:54.304695 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:54.304511    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/70c62fc1-3452-47f0-8d1f-5495e535cd95-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/70c62fc1-3452-47f0-8d1f-5495e535cd95-istio-data\") pod \"70c62fc1-3452-47f0-8d1f-5495e535cd95\" (UID: \"70c62fc1-3452-47f0-8d1f-5495e535cd95\") "
Oct 30 13:20:54.304695 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:54.304574    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/70c62fc1-3452-47f0-8d1f-5495e535cd95-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/70c62fc1-3452-47f0-8d1f-5495e535cd95-credential-socket\") pod \"70c62fc1-3452-47f0-8d1f-5495e535cd95\" (UID: \"70c62fc1-3452-47f0-8d1f-5495e535cd95\") "
Oct 30 13:20:54.305586 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:54.305087    4943 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cf47ace6-4db7-4896-98a4-496dab0e722f-istio-data\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:20:54.305586 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:54.305111    4943 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zp2tx\" (UniqueName: \"kubernetes.io/projected/cf47ace6-4db7-4896-98a4-496dab0e722f-kube-api-access-zp2tx\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:20:54.305586 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:54.305127    4943 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cf47ace6-4db7-4896-98a4-496dab0e722f-istiod-ca-cert\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:20:54.305586 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:54.305142    4943 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cf47ace6-4db7-4896-98a4-496dab0e722f-istio-envoy\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:20:54.305586 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:54.305155    4943 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cf47ace6-4db7-4896-98a4-496dab0e722f-istio-token\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:20:54.305586 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:54.305167    4943 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cf47ace6-4db7-4896-98a4-496dab0e722f-workload-certs\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:20:54.305586 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:54.305179    4943 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cf47ace6-4db7-4896-98a4-496dab0e722f-istio-podinfo\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:20:54.305586 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:54.305190    4943 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf47ace6-4db7-4896-98a4-496dab0e722f-credential-socket\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:20:54.305586 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:54.305202    4943 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf47ace6-4db7-4896-98a4-496dab0e722f-workload-socket\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:20:54.305586 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:54.305501    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/70c62fc1-3452-47f0-8d1f-5495e535cd95-credential-socket" pod "70c62fc1-3452-47f0-8d1f-5495e535cd95" (UID: "70c62fc1-3452-47f0-8d1f-5495e535cd95"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:20:54.307014 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:54.305618    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/70c62fc1-3452-47f0-8d1f-5495e535cd95-workload-certs" pod "70c62fc1-3452-47f0-8d1f-5495e535cd95" (UID: "70c62fc1-3452-47f0-8d1f-5495e535cd95"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:20:54.307014 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:54.305999    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/70c62fc1-3452-47f0-8d1f-5495e535cd95-workload-socket" pod "70c62fc1-3452-47f0-8d1f-5495e535cd95" (UID: "70c62fc1-3452-47f0-8d1f-5495e535cd95"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:20:54.307014 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:54.306384    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/70c62fc1-3452-47f0-8d1f-5495e535cd95-istio-data" pod "70c62fc1-3452-47f0-8d1f-5495e535cd95" (UID: "70c62fc1-3452-47f0-8d1f-5495e535cd95"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:20:54.307014 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:54.306731    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/70c62fc1-3452-47f0-8d1f-5495e535cd95-istiod-ca-cert" pod "70c62fc1-3452-47f0-8d1f-5495e535cd95" (UID: "70c62fc1-3452-47f0-8d1f-5495e535cd95"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 13:20:54.308816 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:54.308752    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/70c62fc1-3452-47f0-8d1f-5495e535cd95-kube-api-access-t87zw" pod "70c62fc1-3452-47f0-8d1f-5495e535cd95" (UID: "70c62fc1-3452-47f0-8d1f-5495e535cd95"). InnerVolumeSpecName "kube-api-access-t87zw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:20:54.312236 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:54.312198    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/70c62fc1-3452-47f0-8d1f-5495e535cd95-istio-token" pod "70c62fc1-3452-47f0-8d1f-5495e535cd95" (UID: "70c62fc1-3452-47f0-8d1f-5495e535cd95"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:20:54.313062 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:54.313026    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/70c62fc1-3452-47f0-8d1f-5495e535cd95-istio-envoy" pod "70c62fc1-3452-47f0-8d1f-5495e535cd95" (UID: "70c62fc1-3452-47f0-8d1f-5495e535cd95"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:20:54.314677 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:54.314641    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/70c62fc1-3452-47f0-8d1f-5495e535cd95-istio-podinfo" pod "70c62fc1-3452-47f0-8d1f-5495e535cd95" (UID: "70c62fc1-3452-47f0-8d1f-5495e535cd95"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 13:20:54.316316 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:54.316296    4943 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-t7zm7"
Oct 30 13:20:54.318407 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:54.318338    4943 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-sfbdb"
Oct 30 13:20:54.406982 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:54.406273    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0ee4232a-d7fe-4aa9-963c-4bac5daa86c3-istiod-ca-cert" pod "0ee4232a-d7fe-4aa9-963c-4bac5daa86c3" (UID: "0ee4232a-d7fe-4aa9-963c-4bac5daa86c3"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 13:20:54.406982 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:54.406342    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0ee4232a-d7fe-4aa9-963c-4bac5daa86c3-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0ee4232a-d7fe-4aa9-963c-4bac5daa86c3-istiod-ca-cert\") pod \"0ee4232a-d7fe-4aa9-963c-4bac5daa86c3\" (UID: \"0ee4232a-d7fe-4aa9-963c-4bac5daa86c3\") "
Oct 30 13:20:54.406982 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:54.406393    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4a720f44-f6a3-48b5-8ba4-04b3214486eb-kube-api-access-xjzlf\" (UniqueName: \"kubernetes.io/projected/4a720f44-f6a3-48b5-8ba4-04b3214486eb-kube-api-access-xjzlf\") pod \"4a720f44-f6a3-48b5-8ba4-04b3214486eb\" (UID: \"4a720f44-f6a3-48b5-8ba4-04b3214486eb\") "
Oct 30 13:20:54.406982 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:54.406452    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0ee4232a-d7fe-4aa9-963c-4bac5daa86c3-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0ee4232a-d7fe-4aa9-963c-4bac5daa86c3-workload-socket\") pod \"0ee4232a-d7fe-4aa9-963c-4bac5daa86c3\" (UID: \"0ee4232a-d7fe-4aa9-963c-4bac5daa86c3\") "
Oct 30 13:20:54.406982 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:54.406844    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0ee4232a-d7fe-4aa9-963c-4bac5daa86c3-kube-api-access-6cvkg\" (UniqueName: \"kubernetes.io/projected/0ee4232a-d7fe-4aa9-963c-4bac5daa86c3-kube-api-access-6cvkg\") pod \"0ee4232a-d7fe-4aa9-963c-4bac5daa86c3\" (UID: \"0ee4232a-d7fe-4aa9-963c-4bac5daa86c3\") "
Oct 30 13:20:54.406982 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:54.406912    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0ee4232a-d7fe-4aa9-963c-4bac5daa86c3-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0ee4232a-d7fe-4aa9-963c-4bac5daa86c3-istio-envoy\") pod \"0ee4232a-d7fe-4aa9-963c-4bac5daa86c3\" (UID: \"0ee4232a-d7fe-4aa9-963c-4bac5daa86c3\") "
Oct 30 13:20:54.406982 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:54.406955    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4a720f44-f6a3-48b5-8ba4-04b3214486eb-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4a720f44-f6a3-48b5-8ba4-04b3214486eb-credential-socket\") pod \"4a720f44-f6a3-48b5-8ba4-04b3214486eb\" (UID: \"4a720f44-f6a3-48b5-8ba4-04b3214486eb\") "
Oct 30 13:20:54.407440 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:54.407242    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0ee4232a-d7fe-4aa9-963c-4bac5daa86c3-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0ee4232a-d7fe-4aa9-963c-4bac5daa86c3-istio-data\") pod \"0ee4232a-d7fe-4aa9-963c-4bac5daa86c3\" (UID: \"0ee4232a-d7fe-4aa9-963c-4bac5daa86c3\") "
Oct 30 13:20:54.407440 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:54.407320    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4a720f44-f6a3-48b5-8ba4-04b3214486eb-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4a720f44-f6a3-48b5-8ba4-04b3214486eb-workload-certs\") pod \"4a720f44-f6a3-48b5-8ba4-04b3214486eb\" (UID: \"4a720f44-f6a3-48b5-8ba4-04b3214486eb\") "
Oct 30 13:20:54.407440 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:54.407322    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0ee4232a-d7fe-4aa9-963c-4bac5daa86c3-workload-socket" pod "0ee4232a-d7fe-4aa9-963c-4bac5daa86c3" (UID: "0ee4232a-d7fe-4aa9-963c-4bac5daa86c3"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:20:54.407968 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:54.407640    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4a720f44-f6a3-48b5-8ba4-04b3214486eb-workload-certs" pod "4a720f44-f6a3-48b5-8ba4-04b3214486eb" (UID: "4a720f44-f6a3-48b5-8ba4-04b3214486eb"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:20:54.407968 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:54.407855    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0ee4232a-d7fe-4aa9-963c-4bac5daa86c3-istio-data" pod "0ee4232a-d7fe-4aa9-963c-4bac5daa86c3" (UID: "0ee4232a-d7fe-4aa9-963c-4bac5daa86c3"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:20:54.407968 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:54.407888    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4a720f44-f6a3-48b5-8ba4-04b3214486eb-credential-socket" pod "4a720f44-f6a3-48b5-8ba4-04b3214486eb" (UID: "4a720f44-f6a3-48b5-8ba4-04b3214486eb"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:20:54.407968 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:54.407898    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4a720f44-f6a3-48b5-8ba4-04b3214486eb-workload-socket" pod "4a720f44-f6a3-48b5-8ba4-04b3214486eb" (UID: "4a720f44-f6a3-48b5-8ba4-04b3214486eb"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:20:54.408250 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:54.407967    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4a720f44-f6a3-48b5-8ba4-04b3214486eb-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4a720f44-f6a3-48b5-8ba4-04b3214486eb-workload-socket\") pod \"4a720f44-f6a3-48b5-8ba4-04b3214486eb\" (UID: \"4a720f44-f6a3-48b5-8ba4-04b3214486eb\") "
Oct 30 13:20:54.408250 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:54.408020    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/4a720f44-f6a3-48b5-8ba4-04b3214486eb-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4a720f44-f6a3-48b5-8ba4-04b3214486eb-istiod-ca-cert\") pod \"4a720f44-f6a3-48b5-8ba4-04b3214486eb\" (UID: \"4a720f44-f6a3-48b5-8ba4-04b3214486eb\") "
Oct 30 13:20:54.408250 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:54.408071    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0ee4232a-d7fe-4aa9-963c-4bac5daa86c3-istio-token\" (UniqueName: \"kubernetes.io/projected/0ee4232a-d7fe-4aa9-963c-4bac5daa86c3-istio-token\") pod \"0ee4232a-d7fe-4aa9-963c-4bac5daa86c3\" (UID: \"0ee4232a-d7fe-4aa9-963c-4bac5daa86c3\") "
Oct 30 13:20:54.408250 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:54.408121    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/0ee4232a-d7fe-4aa9-963c-4bac5daa86c3-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0ee4232a-d7fe-4aa9-963c-4bac5daa86c3-istio-podinfo\") pod \"0ee4232a-d7fe-4aa9-963c-4bac5daa86c3\" (UID: \"0ee4232a-d7fe-4aa9-963c-4bac5daa86c3\") "
Oct 30 13:20:54.408250 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:54.408153    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4a720f44-f6a3-48b5-8ba4-04b3214486eb-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4a720f44-f6a3-48b5-8ba4-04b3214486eb-istio-data\") pod \"4a720f44-f6a3-48b5-8ba4-04b3214486eb\" (UID: \"4a720f44-f6a3-48b5-8ba4-04b3214486eb\") "
Oct 30 13:20:54.408250 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:54.408195    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0ee4232a-d7fe-4aa9-963c-4bac5daa86c3-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0ee4232a-d7fe-4aa9-963c-4bac5daa86c3-credential-socket\") pod \"0ee4232a-d7fe-4aa9-963c-4bac5daa86c3\" (UID: \"0ee4232a-d7fe-4aa9-963c-4bac5daa86c3\") "
Oct 30 13:20:54.408250 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:54.408236    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4a720f44-f6a3-48b5-8ba4-04b3214486eb-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4a720f44-f6a3-48b5-8ba4-04b3214486eb-istio-envoy\") pod \"4a720f44-f6a3-48b5-8ba4-04b3214486eb\" (UID: \"4a720f44-f6a3-48b5-8ba4-04b3214486eb\") "
Oct 30 13:20:54.408615 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:54.408290    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4a720f44-f6a3-48b5-8ba4-04b3214486eb-istio-token\" (UniqueName: \"kubernetes.io/projected/4a720f44-f6a3-48b5-8ba4-04b3214486eb-istio-token\") pod \"4a720f44-f6a3-48b5-8ba4-04b3214486eb\" (UID: \"4a720f44-f6a3-48b5-8ba4-04b3214486eb\") "
Oct 30 13:20:54.408615 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:54.408400    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/4a720f44-f6a3-48b5-8ba4-04b3214486eb-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4a720f44-f6a3-48b5-8ba4-04b3214486eb-istio-podinfo\") pod \"4a720f44-f6a3-48b5-8ba4-04b3214486eb\" (UID: \"4a720f44-f6a3-48b5-8ba4-04b3214486eb\") "
Oct 30 13:20:54.408615 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:54.408504    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0ee4232a-d7fe-4aa9-963c-4bac5daa86c3-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0ee4232a-d7fe-4aa9-963c-4bac5daa86c3-workload-certs\") pod \"0ee4232a-d7fe-4aa9-963c-4bac5daa86c3\" (UID: \"0ee4232a-d7fe-4aa9-963c-4bac5daa86c3\") "
Oct 30 13:20:54.409060 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:54.409039    4943 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/70c62fc1-3452-47f0-8d1f-5495e535cd95-credential-socket\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:20:54.409205 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:54.409063    4943 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0ee4232a-d7fe-4aa9-963c-4bac5daa86c3-istiod-ca-cert\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:20:54.409205 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:54.409078    4943 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0ee4232a-d7fe-4aa9-963c-4bac5daa86c3-workload-socket\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:20:54.409205 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:54.409091    4943 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4a720f44-f6a3-48b5-8ba4-04b3214486eb-credential-socket\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:20:54.409205 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:54.409108    4943 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0ee4232a-d7fe-4aa9-963c-4bac5daa86c3-istio-data\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:20:54.409205 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:54.409124    4943 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4a720f44-f6a3-48b5-8ba4-04b3214486eb-workload-certs\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:20:54.409205 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:54.409138    4943 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t87zw\" (UniqueName: \"kubernetes.io/projected/70c62fc1-3452-47f0-8d1f-5495e535cd95-kube-api-access-t87zw\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:20:54.409205 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:54.409152    4943 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/70c62fc1-3452-47f0-8d1f-5495e535cd95-workload-socket\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:20:54.409205 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:54.409172    4943 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4a720f44-f6a3-48b5-8ba4-04b3214486eb-workload-socket\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:20:54.409205 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:54.409187    4943 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/70c62fc1-3452-47f0-8d1f-5495e535cd95-istiod-ca-cert\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:20:54.409205 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:54.409201    4943 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/70c62fc1-3452-47f0-8d1f-5495e535cd95-istio-envoy\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:20:54.410123 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:54.409214    4943 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/70c62fc1-3452-47f0-8d1f-5495e535cd95-istio-token\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:20:54.410123 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:54.409277    4943 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/70c62fc1-3452-47f0-8d1f-5495e535cd95-istio-podinfo\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:20:54.410123 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:54.409293    4943 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/70c62fc1-3452-47f0-8d1f-5495e535cd95-workload-certs\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:20:54.410123 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:54.409306    4943 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/70c62fc1-3452-47f0-8d1f-5495e535cd95-istio-data\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:20:54.410123 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:54.409530    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0ee4232a-d7fe-4aa9-963c-4bac5daa86c3-workload-certs" pod "0ee4232a-d7fe-4aa9-963c-4bac5daa86c3" (UID: "0ee4232a-d7fe-4aa9-963c-4bac5daa86c3"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:20:54.410123 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:54.409832    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4a720f44-f6a3-48b5-8ba4-04b3214486eb-istiod-ca-cert" pod "4a720f44-f6a3-48b5-8ba4-04b3214486eb" (UID: "4a720f44-f6a3-48b5-8ba4-04b3214486eb"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 13:20:54.410761 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:54.410738    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0ee4232a-d7fe-4aa9-963c-4bac5daa86c3-credential-socket" pod "0ee4232a-d7fe-4aa9-963c-4bac5daa86c3" (UID: "0ee4232a-d7fe-4aa9-963c-4bac5daa86c3"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:20:54.411850 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:54.411821    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0ee4232a-d7fe-4aa9-963c-4bac5daa86c3-istio-envoy" pod "0ee4232a-d7fe-4aa9-963c-4bac5daa86c3" (UID: "0ee4232a-d7fe-4aa9-963c-4bac5daa86c3"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:20:54.412185 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:54.412153    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0ee4232a-d7fe-4aa9-963c-4bac5daa86c3-kube-api-access-6cvkg" pod "0ee4232a-d7fe-4aa9-963c-4bac5daa86c3" (UID: "0ee4232a-d7fe-4aa9-963c-4bac5daa86c3"). InnerVolumeSpecName "kube-api-access-6cvkg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:20:54.412832 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:54.412728    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4a720f44-f6a3-48b5-8ba4-04b3214486eb-istio-data" pod "4a720f44-f6a3-48b5-8ba4-04b3214486eb" (UID: "4a720f44-f6a3-48b5-8ba4-04b3214486eb"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:20:54.413573 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:54.413506    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0ee4232a-d7fe-4aa9-963c-4bac5daa86c3-istio-token" pod "0ee4232a-d7fe-4aa9-963c-4bac5daa86c3" (UID: "0ee4232a-d7fe-4aa9-963c-4bac5daa86c3"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:20:54.414366 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:54.414227    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4a720f44-f6a3-48b5-8ba4-04b3214486eb-kube-api-access-xjzlf" pod "4a720f44-f6a3-48b5-8ba4-04b3214486eb" (UID: "4a720f44-f6a3-48b5-8ba4-04b3214486eb"). InnerVolumeSpecName "kube-api-access-xjzlf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:20:54.415995 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:54.415945    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/0ee4232a-d7fe-4aa9-963c-4bac5daa86c3-istio-podinfo" pod "0ee4232a-d7fe-4aa9-963c-4bac5daa86c3" (UID: "0ee4232a-d7fe-4aa9-963c-4bac5daa86c3"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 13:20:54.418416 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:54.418391    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4a720f44-f6a3-48b5-8ba4-04b3214486eb-istio-token" pod "4a720f44-f6a3-48b5-8ba4-04b3214486eb" (UID: "4a720f44-f6a3-48b5-8ba4-04b3214486eb"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:20:54.418619 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:54.418404    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/4a720f44-f6a3-48b5-8ba4-04b3214486eb-istio-podinfo" pod "4a720f44-f6a3-48b5-8ba4-04b3214486eb" (UID: "4a720f44-f6a3-48b5-8ba4-04b3214486eb"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 13:20:54.418716 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:54.418580    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4a720f44-f6a3-48b5-8ba4-04b3214486eb-istio-envoy" pod "4a720f44-f6a3-48b5-8ba4-04b3214486eb" (UID: "4a720f44-f6a3-48b5-8ba4-04b3214486eb"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:20:54.518180 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:54.517332    4943 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4a720f44-f6a3-48b5-8ba4-04b3214486eb-istiod-ca-cert\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:20:54.518180 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:54.517365    4943 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0ee4232a-d7fe-4aa9-963c-4bac5daa86c3-istio-token\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:20:54.518180 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:54.517374    4943 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0ee4232a-d7fe-4aa9-963c-4bac5daa86c3-istio-podinfo\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:20:54.518180 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:54.517382    4943 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4a720f44-f6a3-48b5-8ba4-04b3214486eb-istio-data\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:20:54.518180 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:54.517391    4943 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0ee4232a-d7fe-4aa9-963c-4bac5daa86c3-credential-socket\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:20:54.518180 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:54.517400    4943 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4a720f44-f6a3-48b5-8ba4-04b3214486eb-istio-envoy\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:20:54.518180 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:54.517408    4943 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4a720f44-f6a3-48b5-8ba4-04b3214486eb-istio-token\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:20:54.518180 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:54.517416    4943 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4a720f44-f6a3-48b5-8ba4-04b3214486eb-istio-podinfo\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:20:54.518180 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:54.517423    4943 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0ee4232a-d7fe-4aa9-963c-4bac5daa86c3-workload-certs\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:20:54.518180 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:54.517432    4943 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xjzlf\" (UniqueName: \"kubernetes.io/projected/4a720f44-f6a3-48b5-8ba4-04b3214486eb-kube-api-access-xjzlf\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:20:54.518180 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:54.517440    4943 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6cvkg\" (UniqueName: \"kubernetes.io/projected/0ee4232a-d7fe-4aa9-963c-4bac5daa86c3-kube-api-access-6cvkg\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:20:54.518180 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:54.517448    4943 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0ee4232a-d7fe-4aa9-963c-4bac5daa86c3-istio-envoy\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:20:54.554921 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:54.554887    4943 generic.go:356] "Generic (PLEG): container finished" podID="70c62fc1-3452-47f0-8d1f-5495e535cd95" containerID="eead86c384c8ed3785b2c45cc493e7003a79c6c27a92ccc315f6cbca01aa5676" exitCode=0
Oct 30 13:20:54.555100 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:54.554958    4943 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-9ndzw"
Oct 30 13:20:54.555100 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:54.554966    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-9ndzw" event={"ID":"70c62fc1-3452-47f0-8d1f-5495e535cd95","Type":"ContainerDied","Data":"eead86c384c8ed3785b2c45cc493e7003a79c6c27a92ccc315f6cbca01aa5676"}
Oct 30 13:20:54.555188 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:54.555103    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-9ndzw" event={"ID":"70c62fc1-3452-47f0-8d1f-5495e535cd95","Type":"ContainerDied","Data":"7edaf6646307190bf2f10077a55ea9a16dbf84b8c86d56980ad39acd7a082e1a"}
Oct 30 13:20:54.555188 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:54.555137    4943 scope.go:122] "RemoveContainer" containerID="eead86c384c8ed3785b2c45cc493e7003a79c6c27a92ccc315f6cbca01aa5676"
Oct 30 13:20:54.557523 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:54.557495    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-586tt" event={"ID":"9c5b6db1-55db-447c-a3a9-2e24a9dc9dc6","Type":"ContainerStarted","Data":"ec6a6067cc5996f4434d40951dca64d038e1e6b22da9fd8cb27397b56344a9d9"}
Oct 30 13:20:54.557713 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:54.557691    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-586tt" event={"ID":"9c5b6db1-55db-447c-a3a9-2e24a9dc9dc6","Type":"ContainerStarted","Data":"24993988e17052f2727f3f8b1cafabeee52d1c253548cb432921cbd5915d4ead"}
Oct 30 13:20:54.559168 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:54.559144    4943 generic.go:356] "Generic (PLEG): container finished" podID="4a720f44-f6a3-48b5-8ba4-04b3214486eb" containerID="a154f39e24a8c3c65170b5213a6d298fbfbe40344cc4164fa7794afbf59fc265" exitCode=0
Oct 30 13:20:54.559307 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:54.559212    4943 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-sfbdb"
Oct 30 13:20:54.559307 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:54.559228    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-sfbdb" event={"ID":"4a720f44-f6a3-48b5-8ba4-04b3214486eb","Type":"ContainerDied","Data":"a154f39e24a8c3c65170b5213a6d298fbfbe40344cc4164fa7794afbf59fc265"}
Oct 30 13:20:54.559307 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:54.559279    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-sfbdb" event={"ID":"4a720f44-f6a3-48b5-8ba4-04b3214486eb","Type":"ContainerDied","Data":"31b96717eacfe5e1fe916de9824de1d0dcef71d9f14dbbc8d6058318f04d8cd1"}
Oct 30 13:20:54.591778 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:54.591715    4943 generic.go:356] "Generic (PLEG): container finished" podID="cf47ace6-4db7-4896-98a4-496dab0e722f" containerID="479a3b20dd15fe9271601fddfd828062fcff28532944df485c2ebee2756c0d76" exitCode=0
Oct 30 13:20:54.592360 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:54.591991    4943 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-2zhxj"
Oct 30 13:20:54.592360 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:54.592330    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-2zhxj" event={"ID":"cf47ace6-4db7-4896-98a4-496dab0e722f","Type":"ContainerDied","Data":"479a3b20dd15fe9271601fddfd828062fcff28532944df485c2ebee2756c0d76"}
Oct 30 13:20:54.592458 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:54.592400    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-2zhxj" event={"ID":"cf47ace6-4db7-4896-98a4-496dab0e722f","Type":"ContainerDied","Data":"045c2b4d07ecf16da58ab760f733bfb124480a45354ae0817b4afd6ee2960c69"}
Oct 30 13:20:54.592573 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:54.592543    4943 scope.go:122] "RemoveContainer" containerID="eead86c384c8ed3785b2c45cc493e7003a79c6c27a92ccc315f6cbca01aa5676"
Oct 30 13:20:54.593380 ip-10-0-142-84 kubenswrapper[4943]: E1030 13:20:54.593347    4943 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eead86c384c8ed3785b2c45cc493e7003a79c6c27a92ccc315f6cbca01aa5676\": container with ID starting with eead86c384c8ed3785b2c45cc493e7003a79c6c27a92ccc315f6cbca01aa5676 not found: ID does not exist" containerID="eead86c384c8ed3785b2c45cc493e7003a79c6c27a92ccc315f6cbca01aa5676"
Oct 30 13:20:54.593517 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:54.593392    4943 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eead86c384c8ed3785b2c45cc493e7003a79c6c27a92ccc315f6cbca01aa5676"} err="failed to get container status \"eead86c384c8ed3785b2c45cc493e7003a79c6c27a92ccc315f6cbca01aa5676\": rpc error: code = NotFound desc = could not find container \"eead86c384c8ed3785b2c45cc493e7003a79c6c27a92ccc315f6cbca01aa5676\": container with ID starting with eead86c384c8ed3785b2c45cc493e7003a79c6c27a92ccc315f6cbca01aa5676 not found: ID does not exist"
Oct 30 13:20:54.593517 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:54.593416    4943 scope.go:122] "RemoveContainer" containerID="a154f39e24a8c3c65170b5213a6d298fbfbe40344cc4164fa7794afbf59fc265"
Oct 30 13:20:54.595666 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:54.595636    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-88f6a4-7tfvz" event={"ID":"5b325a80-09c9-4350-8e01-f50cfab3e2a7","Type":"ContainerDied","Data":"9b08893b3a4475f7ab83be7232f5e522b7a3588b69bb8187741c920ad281829c"}
Oct 30 13:20:54.595766 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:54.595697    4943 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9b08893b3a4475f7ab83be7232f5e522b7a3588b69bb8187741c920ad281829c"
Oct 30 13:20:54.595766 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:54.595601    4943 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-88f6a4-7tfvz"
Oct 30 13:20:54.597943 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:54.597917    4943 generic.go:356] "Generic (PLEG): container finished" podID="0ee4232a-d7fe-4aa9-963c-4bac5daa86c3" containerID="b08612a8773ce4d0dae9d62e24dd8ab0b64f53425ac633e21d4c06ed876f2696" exitCode=0
Oct 30 13:20:54.598332 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:54.597981    4943 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-t7zm7"
Oct 30 13:20:54.598332 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:54.598109    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-t7zm7" event={"ID":"0ee4232a-d7fe-4aa9-963c-4bac5daa86c3","Type":"ContainerDied","Data":"b08612a8773ce4d0dae9d62e24dd8ab0b64f53425ac633e21d4c06ed876f2696"}
Oct 30 13:20:54.598332 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:54.598174    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-t7zm7" event={"ID":"0ee4232a-d7fe-4aa9-963c-4bac5daa86c3","Type":"ContainerDied","Data":"37ccfbf1ae85542fe9d71b4279204f9e06c34e9e1af9f834a11005b3939d87be"}
Oct 30 13:20:54.609060 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:54.609042    4943 scope.go:122] "RemoveContainer" containerID="a154f39e24a8c3c65170b5213a6d298fbfbe40344cc4164fa7794afbf59fc265"
Oct 30 13:20:54.609365 ip-10-0-142-84 kubenswrapper[4943]: E1030 13:20:54.609341    4943 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a154f39e24a8c3c65170b5213a6d298fbfbe40344cc4164fa7794afbf59fc265\": container with ID starting with a154f39e24a8c3c65170b5213a6d298fbfbe40344cc4164fa7794afbf59fc265 not found: ID does not exist" containerID="a154f39e24a8c3c65170b5213a6d298fbfbe40344cc4164fa7794afbf59fc265"
Oct 30 13:20:54.609457 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:54.609378    4943 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a154f39e24a8c3c65170b5213a6d298fbfbe40344cc4164fa7794afbf59fc265"} err="failed to get container status \"a154f39e24a8c3c65170b5213a6d298fbfbe40344cc4164fa7794afbf59fc265\": rpc error: code = NotFound desc = could not find container \"a154f39e24a8c3c65170b5213a6d298fbfbe40344cc4164fa7794afbf59fc265\": container with ID starting with a154f39e24a8c3c65170b5213a6d298fbfbe40344cc4164fa7794afbf59fc265 not found: ID does not exist"
Oct 30 13:20:54.609457 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:54.609401    4943 scope.go:122] "RemoveContainer" containerID="479a3b20dd15fe9271601fddfd828062fcff28532944df485c2ebee2756c0d76"
Oct 30 13:20:54.628792 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:54.628206    4943 scope.go:122] "RemoveContainer" containerID="479a3b20dd15fe9271601fddfd828062fcff28532944df485c2ebee2756c0d76"
Oct 30 13:20:54.628792 ip-10-0-142-84 kubenswrapper[4943]: E1030 13:20:54.628573    4943 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"479a3b20dd15fe9271601fddfd828062fcff28532944df485c2ebee2756c0d76\": container with ID starting with 479a3b20dd15fe9271601fddfd828062fcff28532944df485c2ebee2756c0d76 not found: ID does not exist" containerID="479a3b20dd15fe9271601fddfd828062fcff28532944df485c2ebee2756c0d76"
Oct 30 13:20:54.628792 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:54.628599    4943 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"479a3b20dd15fe9271601fddfd828062fcff28532944df485c2ebee2756c0d76"} err="failed to get container status \"479a3b20dd15fe9271601fddfd828062fcff28532944df485c2ebee2756c0d76\": rpc error: code = NotFound desc = could not find container \"479a3b20dd15fe9271601fddfd828062fcff28532944df485c2ebee2756c0d76\": container with ID starting with 479a3b20dd15fe9271601fddfd828062fcff28532944df485c2ebee2756c0d76 not found: ID does not exist"
Oct 30 13:20:54.628792 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:54.628616    4943 scope.go:122] "RemoveContainer" containerID="b08612a8773ce4d0dae9d62e24dd8ab0b64f53425ac633e21d4c06ed876f2696"
Oct 30 13:20:54.639472 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:54.639447    4943 scope.go:122] "RemoveContainer" containerID="b08612a8773ce4d0dae9d62e24dd8ab0b64f53425ac633e21d4c06ed876f2696"
Oct 30 13:20:54.639747 ip-10-0-142-84 kubenswrapper[4943]: E1030 13:20:54.639727    4943 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b08612a8773ce4d0dae9d62e24dd8ab0b64f53425ac633e21d4c06ed876f2696\": container with ID starting with b08612a8773ce4d0dae9d62e24dd8ab0b64f53425ac633e21d4c06ed876f2696 not found: ID does not exist" containerID="b08612a8773ce4d0dae9d62e24dd8ab0b64f53425ac633e21d4c06ed876f2696"
Oct 30 13:20:54.639819 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:54.639753    4943 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b08612a8773ce4d0dae9d62e24dd8ab0b64f53425ac633e21d4c06ed876f2696"} err="failed to get container status \"b08612a8773ce4d0dae9d62e24dd8ab0b64f53425ac633e21d4c06ed876f2696\": rpc error: code = NotFound desc = could not find container \"b08612a8773ce4d0dae9d62e24dd8ab0b64f53425ac633e21d4c06ed876f2696\": container with ID starting with b08612a8773ce4d0dae9d62e24dd8ab0b64f53425ac633e21d4c06ed876f2696 not found: ID does not exist"
Oct 30 13:20:54.641969 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:54.641943    4943 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-9ndzw"]
Oct 30 13:20:54.651494 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:54.651467    4943 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-9ndzw"]
Oct 30 13:20:54.698235 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:54.698169    4943 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-586tt" podStartSLOduration=2.698148793 podStartE2EDuration="2.698148793s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:20:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:20:54.693969119 +0000 UTC m=+1430.481972850" watchObservedRunningTime="2026-10-30 13:20:54.698148793 +0000 UTC m=+1430.486152525"
Oct 30 13:20:54.804875 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:54.804823    4943 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-2zhxj"]
Oct 30 13:20:54.810935 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:54.810901    4943 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-2zhxj"]
Oct 30 13:20:54.852402 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:54.852369    4943 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-sfbdb"]
Oct 30 13:20:54.861044 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:54.861013    4943 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-sfbdb"]
Oct 30 13:20:54.915402 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:54.915361    4943 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-t7zm7"]
Oct 30 13:20:54.921807 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:54.921779    4943 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-t7zm7"]
Oct 30 13:20:55.016641 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:55.016562    4943 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-e413f738-h9kh9"]
Oct 30 13:20:55.017109 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:55.017093    4943 cpu_manager.go:414] "containerMap: removing container" podUID="5b325a80-09c9-4350-8e01-f50cfab3e2a7" containerName="revoke-keys"
Oct 30 13:20:55.017109 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:55.017109    4943 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5b325a80-09c9-4350-8e01-f50cfab3e2a7" containerName="revoke-keys"
Oct 30 13:20:55.017243 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:55.017122    4943 cpu_manager.go:414] "containerMap: removing container" podUID="4a720f44-f6a3-48b5-8ba4-04b3214486eb" containerName="istio-proxy"
Oct 30 13:20:55.017243 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:55.017127    4943 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4a720f44-f6a3-48b5-8ba4-04b3214486eb" containerName="istio-proxy"
Oct 30 13:20:55.017243 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:55.017165    4943 cpu_manager.go:414] "containerMap: removing container" podUID="0ee4232a-d7fe-4aa9-963c-4bac5daa86c3" containerName="istio-proxy"
Oct 30 13:20:55.017243 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:55.017171    4943 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0ee4232a-d7fe-4aa9-963c-4bac5daa86c3" containerName="istio-proxy"
Oct 30 13:20:55.017243 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:55.017180    4943 cpu_manager.go:414] "containerMap: removing container" podUID="cf47ace6-4db7-4896-98a4-496dab0e722f" containerName="istio-proxy"
Oct 30 13:20:55.017243 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:55.017185    4943 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="cf47ace6-4db7-4896-98a4-496dab0e722f" containerName="istio-proxy"
Oct 30 13:20:55.017243 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:55.017197    4943 cpu_manager.go:414] "containerMap: removing container" podUID="70c62fc1-3452-47f0-8d1f-5495e535cd95" containerName="istio-proxy"
Oct 30 13:20:55.017243 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:55.017201    4943 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="70c62fc1-3452-47f0-8d1f-5495e535cd95" containerName="istio-proxy"
Oct 30 13:20:55.017657 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:55.017297    4943 memory_manager.go:365] "RemoveStaleState removing state" podUID="0ee4232a-d7fe-4aa9-963c-4bac5daa86c3" containerName="istio-proxy"
Oct 30 13:20:55.017657 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:55.017306    4943 memory_manager.go:365] "RemoveStaleState removing state" podUID="cf47ace6-4db7-4896-98a4-496dab0e722f" containerName="istio-proxy"
Oct 30 13:20:55.017657 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:55.017315    4943 memory_manager.go:365] "RemoveStaleState removing state" podUID="70c62fc1-3452-47f0-8d1f-5495e535cd95" containerName="istio-proxy"
Oct 30 13:20:55.017657 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:55.017322    4943 memory_manager.go:365] "RemoveStaleState removing state" podUID="5b325a80-09c9-4350-8e01-f50cfab3e2a7" containerName="revoke-keys"
Oct 30 13:20:55.017657 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:55.017329    4943 memory_manager.go:365] "RemoveStaleState removing state" podUID="4a720f44-f6a3-48b5-8ba4-04b3214486eb" containerName="istio-proxy"
Oct 30 13:20:55.022312 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:55.022292    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-e413f738-h9kh9"
Oct 30 13:20:55.043064 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:55.043037    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-e413f738-h9kh9"]
Oct 30 13:20:55.043488 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:55.043467    4943 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 13:20:55.043632 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:55.043605    4943 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Oct 30 13:20:55.043632 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:55.043509    4943 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Oct 30 13:20:55.043863 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:55.043725    4943 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-p9z57\""
Oct 30 13:20:55.129882 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:55.129841    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/52f8160d-f86d-495e-9e3c-710353788068-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-e413f738-h9kh9\" (UID: \"52f8160d-f86d-495e-9e3c-710353788068\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e413f738-h9kh9"
Oct 30 13:20:55.130164 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:55.130142    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/52f8160d-f86d-495e-9e3c-710353788068-maas-api-xks-ca\") pod \"maas-api-revoke-sub-e413f738-h9kh9\" (UID: \"52f8160d-f86d-495e-9e3c-710353788068\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e413f738-h9kh9"
Oct 30 13:20:55.130398 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:55.130375    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4qwmk\" (UniqueName: \"kubernetes.io/projected/52f8160d-f86d-495e-9e3c-710353788068-kube-api-access-4qwmk\") pod \"maas-api-revoke-sub-e413f738-h9kh9\" (UID: \"52f8160d-f86d-495e-9e3c-710353788068\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e413f738-h9kh9"
Oct 30 13:20:55.190761 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:55.190716    4943 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-88f6a4/distinct-2-w4-88f6a4-kserve-7b8f7bd7dc-5bt68"]
Oct 30 13:20:55.224665 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:55.224631    4943 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-586tt"
Oct 30 13:20:55.226646 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:55.226617    4943 patch_prober.go:29] interesting pod/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-586tt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.126:15021/healthz/ready\": dial tcp 10.132.0.126:15021: connect: connection refused" start-of-body=
Oct 30 13:20:55.226798 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:55.226666    4943 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-586tt" podUID="9c5b6db1-55db-447c-a3a9-2e24a9dc9dc6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.126:15021/healthz/ready\": dial tcp 10.132.0.126:15021: connect: connection refused"
Oct 30 13:20:55.232475 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:55.232439    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/52f8160d-f86d-495e-9e3c-710353788068-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-e413f738-h9kh9\" (UID: \"52f8160d-f86d-495e-9e3c-710353788068\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e413f738-h9kh9"
Oct 30 13:20:55.232651 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:55.232629    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/52f8160d-f86d-495e-9e3c-710353788068-maas-api-xks-ca\") pod \"maas-api-revoke-sub-e413f738-h9kh9\" (UID: \"52f8160d-f86d-495e-9e3c-710353788068\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e413f738-h9kh9"
Oct 30 13:20:55.232776 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:55.232661    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4qwmk\" (UniqueName: \"kubernetes.io/projected/52f8160d-f86d-495e-9e3c-710353788068-kube-api-access-4qwmk\") pod \"maas-api-revoke-sub-e413f738-h9kh9\" (UID: \"52f8160d-f86d-495e-9e3c-710353788068\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e413f738-h9kh9"
Oct 30 13:20:55.233118 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:55.233097    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/52f8160d-f86d-495e-9e3c-710353788068-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-e413f738-h9kh9\" (UID: \"52f8160d-f86d-495e-9e3c-710353788068\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e413f738-h9kh9"
Oct 30 13:20:55.235179 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:55.235161    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/52f8160d-f86d-495e-9e3c-710353788068-maas-api-xks-ca\") pod \"maas-api-revoke-sub-e413f738-h9kh9\" (UID: \"52f8160d-f86d-495e-9e3c-710353788068\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e413f738-h9kh9"
Oct 30 13:20:55.290905 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:55.290826    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4qwmk\" (UniqueName: \"kubernetes.io/projected/52f8160d-f86d-495e-9e3c-710353788068-kube-api-access-4qwmk\") pod \"maas-api-revoke-sub-e413f738-h9kh9\" (UID: \"52f8160d-f86d-495e-9e3c-710353788068\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e413f738-h9kh9"
Oct 30 13:20:55.341211 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:55.341182    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-e413f738-h9kh9"
Oct 30 13:20:55.696964 ip-10-0-142-84 kubenswrapper[4943]: W1030 13:20:55.696926    4943 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod52f8160d_f86d_495e_9e3c_710353788068.slice/crio-36f88df3a2bcb7f2c2cd62e6e452269baf733c70ff9e93e03ad7bbe9966bfa06 WatchSource:0}: Error finding container 36f88df3a2bcb7f2c2cd62e6e452269baf733c70ff9e93e03ad7bbe9966bfa06: Status 404 returned error can't find the container with id 36f88df3a2bcb7f2c2cd62e6e452269baf733c70ff9e93e03ad7bbe9966bfa06
Oct 30 13:20:55.698668 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:55.698638    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-e413f738-h9kh9"]
Oct 30 13:20:56.151858 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:56.151797    4943 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-w2-f76c09-588d998d5b-z94mv" podUID="278a0939-7f8a-4551-bac6-a4ef6b5f44a9" containerName="payload-processing" containerID="cri-o://961bb93619aa61ec76505cd093319d4b9f87decb2de092205a0d403e678f5366" gracePeriod=25
Oct 30 13:20:56.224818 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:56.224782    4943 patch_prober.go:29] interesting pod/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-586tt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.126:15021/healthz/ready\": dial tcp 10.132.0.126:15021: connect: connection refused" start-of-body=
Oct 30 13:20:56.224988 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:56.224861    4943 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-586tt" podUID="9c5b6db1-55db-447c-a3a9-2e24a9dc9dc6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.126:15021/healthz/ready\": dial tcp 10.132.0.126:15021: connect: connection refused"
Oct 30 13:20:56.225863 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:56.225831    4943 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-f76c09-5bdd4bf999-gbp8q" podUID="f35ce2f9-b769-4ec4-9c57-a31057339d93" containerName="payload-pre-processing" containerID="cri-o://74da8e5e76cf9d27e72eb45e48ad35c66ed936f0c10713c1c015d00666116b29" gracePeriod=25
Oct 30 13:20:56.351316 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:56.351228    4943 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0ee4232a-d7fe-4aa9-963c-4bac5daa86c3" path="/var/lib/kubelet/pods/0ee4232a-d7fe-4aa9-963c-4bac5daa86c3/volumes"
Oct 30 13:20:56.351824 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:56.351805    4943 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4a720f44-f6a3-48b5-8ba4-04b3214486eb" path="/var/lib/kubelet/pods/4a720f44-f6a3-48b5-8ba4-04b3214486eb/volumes"
Oct 30 13:20:56.352308 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:56.352294    4943 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="70c62fc1-3452-47f0-8d1f-5495e535cd95" path="/var/lib/kubelet/pods/70c62fc1-3452-47f0-8d1f-5495e535cd95/volumes"
Oct 30 13:20:56.353123 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:56.353104    4943 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="cf47ace6-4db7-4896-98a4-496dab0e722f" path="/var/lib/kubelet/pods/cf47ace6-4db7-4896-98a4-496dab0e722f/volumes"
Oct 30 13:20:56.628644 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:56.628549    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-e413f738-h9kh9" event={"ID":"52f8160d-f86d-495e-9e3c-710353788068","Type":"ContainerStarted","Data":"e1ddf29adc650c3523d96ebdb4d5b3450dea18f60f4cee4b5fcbca2e8a8b2253"}
Oct 30 13:20:56.628644 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:56.628611    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-e413f738-h9kh9" event={"ID":"52f8160d-f86d-495e-9e3c-710353788068","Type":"ContainerStarted","Data":"36f88df3a2bcb7f2c2cd62e6e452269baf733c70ff9e93e03ad7bbe9966bfa06"}
Oct 30 13:20:56.688791 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:56.688729    4943 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-e413f738-h9kh9" podStartSLOduration=2.688714936 podStartE2EDuration="2.688714936s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:20:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:20:56.675767709 +0000 UTC m=+1432.463771440" watchObservedRunningTime="2026-10-30 13:20:56.688714936 +0000 UTC m=+1432.476718666"
Oct 30 13:20:57.110243 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:57.110190    4943 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w2-f76c09-588d998d5b-z94mv" podUID="278a0939-7f8a-4551-bac6-a4ef6b5f44a9" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 13:20:57.110846 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:57.110823    4943 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-f76c09-5bdd4bf999-gbp8q" podUID="f35ce2f9-b769-4ec4-9c57-a31057339d93" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 13:20:57.224305 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:57.224143    4943 patch_prober.go:29] interesting pod/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-586tt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.126:15021/healthz/ready\": dial tcp 10.132.0.126:15021: connect: connection refused" start-of-body=
Oct 30 13:20:57.224305 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:57.224198    4943 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-586tt" podUID="9c5b6db1-55db-447c-a3a9-2e24a9dc9dc6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.126:15021/healthz/ready\": dial tcp 10.132.0.126:15021: connect: connection refused"
Oct 30 13:20:57.638538 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:57.638503    4943 generic.go:356] "Generic (PLEG): container finished" podID="52f8160d-f86d-495e-9e3c-710353788068" containerID="e1ddf29adc650c3523d96ebdb4d5b3450dea18f60f4cee4b5fcbca2e8a8b2253" exitCode=0
Oct 30 13:20:57.638719 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:57.638586    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-e413f738-h9kh9" event={"ID":"52f8160d-f86d-495e-9e3c-710353788068","Type":"ContainerDied","Data":"e1ddf29adc650c3523d96ebdb4d5b3450dea18f60f4cee4b5fcbca2e8a8b2253"}
Oct 30 13:20:58.224891 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:58.224849    4943 patch_prober.go:29] interesting pod/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-586tt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.126:15021/healthz/ready\": dial tcp 10.132.0.126:15021: connect: connection refused" start-of-body=
Oct 30 13:20:58.225296 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:58.224942    4943 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-586tt" podUID="9c5b6db1-55db-447c-a3a9-2e24a9dc9dc6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.126:15021/healthz/ready\": dial tcp 10.132.0.126:15021: connect: connection refused"
Oct 30 13:20:58.920454 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:58.920432    4943 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-e413f738-h9kh9"
Oct 30 13:20:59.019153 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:59.019114    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/52f8160d-f86d-495e-9e3c-710353788068-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/52f8160d-f86d-495e-9e3c-710353788068-maas-api-openshift-ca\") pod \"52f8160d-f86d-495e-9e3c-710353788068\" (UID: \"52f8160d-f86d-495e-9e3c-710353788068\") "
Oct 30 13:20:59.019339 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:59.019249    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/52f8160d-f86d-495e-9e3c-710353788068-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/52f8160d-f86d-495e-9e3c-710353788068-maas-api-xks-ca\") pod \"52f8160d-f86d-495e-9e3c-710353788068\" (UID: \"52f8160d-f86d-495e-9e3c-710353788068\") "
Oct 30 13:20:59.019421 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:59.019363    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/52f8160d-f86d-495e-9e3c-710353788068-kube-api-access-4qwmk\" (UniqueName: \"kubernetes.io/projected/52f8160d-f86d-495e-9e3c-710353788068-kube-api-access-4qwmk\") pod \"52f8160d-f86d-495e-9e3c-710353788068\" (UID: \"52f8160d-f86d-495e-9e3c-710353788068\") "
Oct 30 13:20:59.019894 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:59.019851    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/52f8160d-f86d-495e-9e3c-710353788068-maas-api-openshift-ca" pod "52f8160d-f86d-495e-9e3c-710353788068" (UID: "52f8160d-f86d-495e-9e3c-710353788068"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 13:20:59.022442 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:59.022409    4943 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/52f8160d-f86d-495e-9e3c-710353788068-maas-api-openshift-ca\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:20:59.029478 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:59.029445    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/52f8160d-f86d-495e-9e3c-710353788068-maas-api-xks-ca" pod "52f8160d-f86d-495e-9e3c-710353788068" (UID: "52f8160d-f86d-495e-9e3c-710353788068"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 13:20:59.031686 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:59.031656    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/52f8160d-f86d-495e-9e3c-710353788068-kube-api-access-4qwmk" pod "52f8160d-f86d-495e-9e3c-710353788068" (UID: "52f8160d-f86d-495e-9e3c-710353788068"). InnerVolumeSpecName "kube-api-access-4qwmk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:20:59.055907 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:59.055884    4943 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-301cef69-8lmsm"]
Oct 30 13:20:59.059285 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:59.059251    4943 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-301cef69-8lmsm"]
Oct 30 13:20:59.123690 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:59.123657    4943 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/52f8160d-f86d-495e-9e3c-710353788068-maas-api-xks-ca\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:20:59.123690 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:59.123689    4943 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4qwmk\" (UniqueName: \"kubernetes.io/projected/52f8160d-f86d-495e-9e3c-710353788068-kube-api-access-4qwmk\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:20:59.224193 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:59.224158    4943 patch_prober.go:29] interesting pod/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-586tt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.126:15021/healthz/ready\": dial tcp 10.132.0.126:15021: connect: connection refused" start-of-body=
Oct 30 13:20:59.224390 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:59.224222    4943 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-586tt" podUID="9c5b6db1-55db-447c-a3a9-2e24a9dc9dc6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.126:15021/healthz/ready\": dial tcp 10.132.0.126:15021: connect: connection refused"
Oct 30 13:20:59.655349 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:59.655250    4943 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-e413f738-h9kh9"
Oct 30 13:20:59.655349 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:59.655295    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-e413f738-h9kh9" event={"ID":"52f8160d-f86d-495e-9e3c-710353788068","Type":"ContainerDied","Data":"36f88df3a2bcb7f2c2cd62e6e452269baf733c70ff9e93e03ad7bbe9966bfa06"}
Oct 30 13:20:59.655349 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:20:59.655348    4943 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="36f88df3a2bcb7f2c2cd62e6e452269baf733c70ff9e93e03ad7bbe9966bfa06"
Oct 30 13:21:00.224795 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:00.224759    4943 patch_prober.go:29] interesting pod/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-586tt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.126:15021/healthz/ready\": dial tcp 10.132.0.126:15021: connect: connection refused" start-of-body=
Oct 30 13:21:00.224980 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:00.224839    4943 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-586tt" podUID="9c5b6db1-55db-447c-a3a9-2e24a9dc9dc6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.126:15021/healthz/ready\": dial tcp 10.132.0.126:15021: connect: connection refused"
Oct 30 13:21:00.352700 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:00.352669    4943 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ea5a5d3f-1175-4fe1-bc37-acf4dcc76a62" path="/var/lib/kubelet/pods/ea5a5d3f-1175-4fe1-bc37-acf4dcc76a62/volumes"
Oct 30 13:21:00.706836 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:00.706776    4943 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w2-f76c09/distinct-2-w2-f76c09-kserve-78446dcc4c-7ljvf" podUID="9bcea043-6bca-43ef-8aa8-f4a82cc98b1a" containerName="main" containerID="cri-o://561eb3fc9f77813de62ee57c5cd76b786f814901f9693f9945837a4ed517f533" gracePeriod=45
Oct 30 13:21:00.825192 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:00.825163    4943 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-f76c09/distinct-2-w2-f76c09-kserve-78446dcc4c-7ljvf" podUID="9bcea043-6bca-43ef-8aa8-f4a82cc98b1a" containerName="main" probeResult="failure" output="Get \"https://10.132.0.120:8000/ready\": dial tcp 10.132.0.120:8000: connect: connection refused"
Oct 30 13:21:01.002524 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:01.002487    4943 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-88f6a4-7dd558b944-gkzck"]
Oct 30 13:21:01.005785 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:01.005758    4943 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-f76c09/distinct-2-w2-f76c09-kserve-78446dcc4c-7ljvf"
Oct 30 13:21:01.124255 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:01.124219    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-hdmrm"]
Oct 30 13:21:01.146559 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:01.146527    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/9bcea043-6bca-43ef-8aa8-f4a82cc98b1a-tls-certs\" (UniqueName: \"kubernetes.io/secret/9bcea043-6bca-43ef-8aa8-f4a82cc98b1a-tls-certs\") pod \"9bcea043-6bca-43ef-8aa8-f4a82cc98b1a\" (UID: \"9bcea043-6bca-43ef-8aa8-f4a82cc98b1a\") "
Oct 30 13:21:01.146729 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:01.146688    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9bcea043-6bca-43ef-8aa8-f4a82cc98b1a-home\" (UniqueName: \"kubernetes.io/empty-dir/9bcea043-6bca-43ef-8aa8-f4a82cc98b1a-home\") pod \"9bcea043-6bca-43ef-8aa8-f4a82cc98b1a\" (UID: \"9bcea043-6bca-43ef-8aa8-f4a82cc98b1a\") "
Oct 30 13:21:01.146815 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:01.146760    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9bcea043-6bca-43ef-8aa8-f4a82cc98b1a-dshm\" (UniqueName: \"kubernetes.io/empty-dir/9bcea043-6bca-43ef-8aa8-f4a82cc98b1a-dshm\") pod \"9bcea043-6bca-43ef-8aa8-f4a82cc98b1a\" (UID: \"9bcea043-6bca-43ef-8aa8-f4a82cc98b1a\") "
Oct 30 13:21:01.146880 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:01.146842    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9bcea043-6bca-43ef-8aa8-f4a82cc98b1a-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9bcea043-6bca-43ef-8aa8-f4a82cc98b1a-tmp-dir\") pod \"9bcea043-6bca-43ef-8aa8-f4a82cc98b1a\" (UID: \"9bcea043-6bca-43ef-8aa8-f4a82cc98b1a\") "
Oct 30 13:21:01.146942 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:01.146925    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9bcea043-6bca-43ef-8aa8-f4a82cc98b1a-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9bcea043-6bca-43ef-8aa8-f4a82cc98b1a-model-cache\") pod \"9bcea043-6bca-43ef-8aa8-f4a82cc98b1a\" (UID: \"9bcea043-6bca-43ef-8aa8-f4a82cc98b1a\") "
Oct 30 13:21:01.147002 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:01.146962    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9bcea043-6bca-43ef-8aa8-f4a82cc98b1a-kube-api-access-c8zbd\" (UniqueName: \"kubernetes.io/projected/9bcea043-6bca-43ef-8aa8-f4a82cc98b1a-kube-api-access-c8zbd\") pod \"9bcea043-6bca-43ef-8aa8-f4a82cc98b1a\" (UID: \"9bcea043-6bca-43ef-8aa8-f4a82cc98b1a\") "
Oct 30 13:21:01.147696 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:01.147543    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9bcea043-6bca-43ef-8aa8-f4a82cc98b1a-tmp-dir" pod "9bcea043-6bca-43ef-8aa8-f4a82cc98b1a" (UID: "9bcea043-6bca-43ef-8aa8-f4a82cc98b1a"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:21:01.147881 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:01.147844    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9bcea043-6bca-43ef-8aa8-f4a82cc98b1a-model-cache" pod "9bcea043-6bca-43ef-8aa8-f4a82cc98b1a" (UID: "9bcea043-6bca-43ef-8aa8-f4a82cc98b1a"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:21:01.148611 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:01.148583    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9bcea043-6bca-43ef-8aa8-f4a82cc98b1a-home" pod "9bcea043-6bca-43ef-8aa8-f4a82cc98b1a" (UID: "9bcea043-6bca-43ef-8aa8-f4a82cc98b1a"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:21:01.148798 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:01.148780    4943 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9bcea043-6bca-43ef-8aa8-f4a82cc98b1a-home\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:21:01.148876 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:01.148803    4943 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9bcea043-6bca-43ef-8aa8-f4a82cc98b1a-tmp-dir\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:21:01.148876 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:01.148817    4943 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9bcea043-6bca-43ef-8aa8-f4a82cc98b1a-model-cache\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:21:01.149616 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:01.149588    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9bcea043-6bca-43ef-8aa8-f4a82cc98b1a-dshm" pod "9bcea043-6bca-43ef-8aa8-f4a82cc98b1a" (UID: "9bcea043-6bca-43ef-8aa8-f4a82cc98b1a"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:21:01.150831 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:01.150801    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9bcea043-6bca-43ef-8aa8-f4a82cc98b1a-tls-certs" pod "9bcea043-6bca-43ef-8aa8-f4a82cc98b1a" (UID: "9bcea043-6bca-43ef-8aa8-f4a82cc98b1a"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 13:21:01.150941 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:01.150889    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9bcea043-6bca-43ef-8aa8-f4a82cc98b1a-kube-api-access-c8zbd" pod "9bcea043-6bca-43ef-8aa8-f4a82cc98b1a" (UID: "9bcea043-6bca-43ef-8aa8-f4a82cc98b1a"). InnerVolumeSpecName "kube-api-access-c8zbd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:21:01.250626 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:01.250551    4943 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9bcea043-6bca-43ef-8aa8-f4a82cc98b1a-dshm\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:21:01.250626 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:01.250584    4943 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c8zbd\" (UniqueName: \"kubernetes.io/projected/9bcea043-6bca-43ef-8aa8-f4a82cc98b1a-kube-api-access-c8zbd\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:21:01.250626 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:01.250599    4943 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9bcea043-6bca-43ef-8aa8-f4a82cc98b1a-tls-certs\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:21:01.303824 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:01.303774    4943 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-586tt"
Oct 30 13:21:01.305196 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:01.305110    4943 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-586tt"
Oct 30 13:21:01.305789 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:01.305763    4943 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-586tt"
Oct 30 13:21:01.673499 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:01.673411    4943 generic.go:356] "Generic (PLEG): container finished" podID="9bcea043-6bca-43ef-8aa8-f4a82cc98b1a" containerID="561eb3fc9f77813de62ee57c5cd76b786f814901f9693f9945837a4ed517f533" exitCode=0
Oct 30 13:21:01.673499 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:01.673465    4943 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-f76c09/distinct-2-w2-f76c09-kserve-78446dcc4c-7ljvf"
Oct 30 13:21:01.673767 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:01.673499    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-f76c09/distinct-2-w2-f76c09-kserve-78446dcc4c-7ljvf" event={"ID":"9bcea043-6bca-43ef-8aa8-f4a82cc98b1a","Type":"ContainerDied","Data":"561eb3fc9f77813de62ee57c5cd76b786f814901f9693f9945837a4ed517f533"}
Oct 30 13:21:01.673767 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:01.673537    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-f76c09/distinct-2-w2-f76c09-kserve-78446dcc4c-7ljvf" event={"ID":"9bcea043-6bca-43ef-8aa8-f4a82cc98b1a","Type":"ContainerDied","Data":"014e15ea14cf1178b9405848957a09008ebbf444de422fd1e15b826e4f74c6fd"}
Oct 30 13:21:01.673767 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:01.673560    4943 scope.go:122] "RemoveContainer" containerID="561eb3fc9f77813de62ee57c5cd76b786f814901f9693f9945837a4ed517f533"
Oct 30 13:21:01.688403 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:01.688384    4943 scope.go:122] "RemoveContainer" containerID="561eb3fc9f77813de62ee57c5cd76b786f814901f9693f9945837a4ed517f533"
Oct 30 13:21:01.689276 ip-10-0-142-84 kubenswrapper[4943]: E1030 13:21:01.689235    4943 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"561eb3fc9f77813de62ee57c5cd76b786f814901f9693f9945837a4ed517f533\": container with ID starting with 561eb3fc9f77813de62ee57c5cd76b786f814901f9693f9945837a4ed517f533 not found: ID does not exist" containerID="561eb3fc9f77813de62ee57c5cd76b786f814901f9693f9945837a4ed517f533"
Oct 30 13:21:01.689587 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:01.689561    4943 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"561eb3fc9f77813de62ee57c5cd76b786f814901f9693f9945837a4ed517f533"} err="failed to get container status \"561eb3fc9f77813de62ee57c5cd76b786f814901f9693f9945837a4ed517f533\": rpc error: code = NotFound desc = could not find container \"561eb3fc9f77813de62ee57c5cd76b786f814901f9693f9945837a4ed517f533\": container with ID starting with 561eb3fc9f77813de62ee57c5cd76b786f814901f9693f9945837a4ed517f533 not found: ID does not exist"
Oct 30 13:21:01.739025 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:01.739002    4943 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w2-f76c09/distinct-2-w2-f76c09-kserve-78446dcc4c-7ljvf"]
Oct 30 13:21:01.746590 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:01.746568    4943 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w2-f76c09/distinct-2-w2-f76c09-kserve-78446dcc4c-7ljvf"]
Oct 30 13:21:02.352949 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:02.352914    4943 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9bcea043-6bca-43ef-8aa8-f4a82cc98b1a" path="/var/lib/kubelet/pods/9bcea043-6bca-43ef-8aa8-f4a82cc98b1a/volumes"
Oct 30 13:21:03.799022 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:03.798988    4943 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-c0df5bf0-2l5v5"]
Oct 30 13:21:03.799506 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:03.799492    4943 cpu_manager.go:414] "containerMap: removing container" podUID="52f8160d-f86d-495e-9e3c-710353788068" containerName="revoke-keys"
Oct 30 13:21:03.799554 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:03.799506    4943 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="52f8160d-f86d-495e-9e3c-710353788068" containerName="revoke-keys"
Oct 30 13:21:03.799554 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:03.799530    4943 cpu_manager.go:414] "containerMap: removing container" podUID="9bcea043-6bca-43ef-8aa8-f4a82cc98b1a" containerName="main"
Oct 30 13:21:03.799554 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:03.799536    4943 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9bcea043-6bca-43ef-8aa8-f4a82cc98b1a" containerName="main"
Oct 30 13:21:03.799663 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:03.799653    4943 memory_manager.go:365] "RemoveStaleState removing state" podUID="9bcea043-6bca-43ef-8aa8-f4a82cc98b1a" containerName="main"
Oct 30 13:21:03.799694 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:03.799670    4943 memory_manager.go:365] "RemoveStaleState removing state" podUID="52f8160d-f86d-495e-9e3c-710353788068" containerName="revoke-keys"
Oct 30 13:21:03.804419 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:03.804399    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c0df5bf0-2l5v5"
Oct 30 13:21:03.826503 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:03.826469    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-c0df5bf0-2l5v5"]
Oct 30 13:21:03.837411 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:03.837390    4943 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 13:21:03.837500 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:03.837428    4943 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-p9z57\""
Oct 30 13:21:03.848775 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:03.848751    4943 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Oct 30 13:21:03.848941 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:03.848878    4943 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Oct 30 13:21:03.890777 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:03.890754    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/bd28f696-44c5-49ef-ac4b-9b63e9b7e5bb-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-c0df5bf0-2l5v5\" (UID: \"bd28f696-44c5-49ef-ac4b-9b63e9b7e5bb\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c0df5bf0-2l5v5"
Oct 30 13:21:03.890876 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:03.890786    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fm478\" (UniqueName: \"kubernetes.io/projected/bd28f696-44c5-49ef-ac4b-9b63e9b7e5bb-kube-api-access-fm478\") pod \"maas-api-revoke-sub-c0df5bf0-2l5v5\" (UID: \"bd28f696-44c5-49ef-ac4b-9b63e9b7e5bb\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c0df5bf0-2l5v5"
Oct 30 13:21:03.890966 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:03.890947    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/bd28f696-44c5-49ef-ac4b-9b63e9b7e5bb-maas-api-xks-ca\") pod \"maas-api-revoke-sub-c0df5bf0-2l5v5\" (UID: \"bd28f696-44c5-49ef-ac4b-9b63e9b7e5bb\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c0df5bf0-2l5v5"
Oct 30 13:21:03.991788 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:03.991760    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/bd28f696-44c5-49ef-ac4b-9b63e9b7e5bb-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-c0df5bf0-2l5v5\" (UID: \"bd28f696-44c5-49ef-ac4b-9b63e9b7e5bb\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c0df5bf0-2l5v5"
Oct 30 13:21:03.991788 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:03.991792    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fm478\" (UniqueName: \"kubernetes.io/projected/bd28f696-44c5-49ef-ac4b-9b63e9b7e5bb-kube-api-access-fm478\") pod \"maas-api-revoke-sub-c0df5bf0-2l5v5\" (UID: \"bd28f696-44c5-49ef-ac4b-9b63e9b7e5bb\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c0df5bf0-2l5v5"
Oct 30 13:21:03.991992 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:03.991882    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/bd28f696-44c5-49ef-ac4b-9b63e9b7e5bb-maas-api-xks-ca\") pod \"maas-api-revoke-sub-c0df5bf0-2l5v5\" (UID: \"bd28f696-44c5-49ef-ac4b-9b63e9b7e5bb\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c0df5bf0-2l5v5"
Oct 30 13:21:03.992429 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:03.992406    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/bd28f696-44c5-49ef-ac4b-9b63e9b7e5bb-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-c0df5bf0-2l5v5\" (UID: \"bd28f696-44c5-49ef-ac4b-9b63e9b7e5bb\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c0df5bf0-2l5v5"
Oct 30 13:21:03.994222 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:03.994206    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/bd28f696-44c5-49ef-ac4b-9b63e9b7e5bb-maas-api-xks-ca\") pod \"maas-api-revoke-sub-c0df5bf0-2l5v5\" (UID: \"bd28f696-44c5-49ef-ac4b-9b63e9b7e5bb\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c0df5bf0-2l5v5"
Oct 30 13:21:04.019563 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:04.019538    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fm478\" (UniqueName: \"kubernetes.io/projected/bd28f696-44c5-49ef-ac4b-9b63e9b7e5bb-kube-api-access-fm478\") pod \"maas-api-revoke-sub-c0df5bf0-2l5v5\" (UID: \"bd28f696-44c5-49ef-ac4b-9b63e9b7e5bb\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c0df5bf0-2l5v5"
Oct 30 13:21:04.118588 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:04.118532    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c0df5bf0-2l5v5"
Oct 30 13:21:04.436988 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:04.436964    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-c0df5bf0-2l5v5"]
Oct 30 13:21:04.439092 ip-10-0-142-84 kubenswrapper[4943]: W1030 13:21:04.439064    4943 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbd28f696_44c5_49ef_ac4b_9b63e9b7e5bb.slice/crio-5ec4624a6d71b069467a9c75410b206884bdbe137ffc3636f8b6fa759601336d WatchSource:0}: Error finding container 5ec4624a6d71b069467a9c75410b206884bdbe137ffc3636f8b6fa759601336d: Status 404 returned error can't find the container with id 5ec4624a6d71b069467a9c75410b206884bdbe137ffc3636f8b6fa759601336d
Oct 30 13:21:04.704249 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:04.704183    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c0df5bf0-2l5v5" event={"ID":"bd28f696-44c5-49ef-ac4b-9b63e9b7e5bb","Type":"ContainerStarted","Data":"5ec4624a6d71b069467a9c75410b206884bdbe137ffc3636f8b6fa759601336d"}
Oct 30 13:21:05.713454 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:05.713417    4943 generic.go:356] "Generic (PLEG): container finished" podID="bd28f696-44c5-49ef-ac4b-9b63e9b7e5bb" containerID="4d3ffa74191f55ca5667753b04d7d786f74eb1d290bba112f293fc4931fcfc12" exitCode=0
Oct 30 13:21:05.713906 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:05.713505    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c0df5bf0-2l5v5" event={"ID":"bd28f696-44c5-49ef-ac4b-9b63e9b7e5bb","Type":"ContainerDied","Data":"4d3ffa74191f55ca5667753b04d7d786f74eb1d290bba112f293fc4931fcfc12"}
Oct 30 13:21:05.955796 ip-10-0-142-84 kubenswrapper[4943]: E1030 13:21:05.955763    4943 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-knqz6" containerName="extractor"
Oct 30 13:21:06.002852 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:06.002773    4943 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-w4-88f6a4-7dd558b944-gkzck" podUID="80c84ecd-0a11-4d8e-ab46-2ba9efd4e676" containerName="payload-processing" containerID="cri-o://bd85b10273b50dd2fca89ae5428adcff4261b0824c739153bd1148b75a648e9f" gracePeriod=25
Oct 30 13:21:06.874709 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:06.874679    4943 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c0df5bf0-2l5v5"
Oct 30 13:21:07.036672 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:07.036592    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/bd28f696-44c5-49ef-ac4b-9b63e9b7e5bb-kube-api-access-fm478\" (UniqueName: \"kubernetes.io/projected/bd28f696-44c5-49ef-ac4b-9b63e9b7e5bb-kube-api-access-fm478\") pod \"bd28f696-44c5-49ef-ac4b-9b63e9b7e5bb\" (UID: \"bd28f696-44c5-49ef-ac4b-9b63e9b7e5bb\") "
Oct 30 13:21:07.036672 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:07.036633    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/bd28f696-44c5-49ef-ac4b-9b63e9b7e5bb-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/bd28f696-44c5-49ef-ac4b-9b63e9b7e5bb-maas-api-openshift-ca\") pod \"bd28f696-44c5-49ef-ac4b-9b63e9b7e5bb\" (UID: \"bd28f696-44c5-49ef-ac4b-9b63e9b7e5bb\") "
Oct 30 13:21:07.036869 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:07.036840    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/bd28f696-44c5-49ef-ac4b-9b63e9b7e5bb-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/bd28f696-44c5-49ef-ac4b-9b63e9b7e5bb-maas-api-xks-ca\") pod \"bd28f696-44c5-49ef-ac4b-9b63e9b7e5bb\" (UID: \"bd28f696-44c5-49ef-ac4b-9b63e9b7e5bb\") "
Oct 30 13:21:07.037090 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:07.037054    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bd28f696-44c5-49ef-ac4b-9b63e9b7e5bb-maas-api-openshift-ca" pod "bd28f696-44c5-49ef-ac4b-9b63e9b7e5bb" (UID: "bd28f696-44c5-49ef-ac4b-9b63e9b7e5bb"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 13:21:07.037382 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:07.037352    4943 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/bd28f696-44c5-49ef-ac4b-9b63e9b7e5bb-maas-api-openshift-ca\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:21:07.038823 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:07.038793    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bd28f696-44c5-49ef-ac4b-9b63e9b7e5bb-kube-api-access-fm478" pod "bd28f696-44c5-49ef-ac4b-9b63e9b7e5bb" (UID: "bd28f696-44c5-49ef-ac4b-9b63e9b7e5bb"). InnerVolumeSpecName "kube-api-access-fm478". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:21:07.038823 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:07.038804    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bd28f696-44c5-49ef-ac4b-9b63e9b7e5bb-maas-api-xks-ca" pod "bd28f696-44c5-49ef-ac4b-9b63e9b7e5bb" (UID: "bd28f696-44c5-49ef-ac4b-9b63e9b7e5bb"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 13:21:07.110655 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:07.110616    4943 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-f76c09-5bdd4bf999-gbp8q" podUID="f35ce2f9-b769-4ec4-9c57-a31057339d93" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 13:21:07.110794 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:07.110616    4943 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w2-f76c09-588d998d5b-z94mv" podUID="278a0939-7f8a-4551-bac6-a4ef6b5f44a9" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 13:21:07.138392 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:07.138366    4943 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/bd28f696-44c5-49ef-ac4b-9b63e9b7e5bb-maas-api-xks-ca\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:21:07.138392 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:07.138390    4943 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fm478\" (UniqueName: \"kubernetes.io/projected/bd28f696-44c5-49ef-ac4b-9b63e9b7e5bb-kube-api-access-fm478\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:21:07.309466 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:07.309381    4943 scope.go:122] "RemoveContainer" containerID="fa5db13b4de531d7eedb9007c9a86fac2d62d0ea1210d17f699bb2e10fc7aa60"
Oct 30 13:21:07.729727 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:07.729695    4943 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c0df5bf0-2l5v5"
Oct 30 13:21:07.729932 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:07.729726    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c0df5bf0-2l5v5" event={"ID":"bd28f696-44c5-49ef-ac4b-9b63e9b7e5bb","Type":"ContainerDied","Data":"5ec4624a6d71b069467a9c75410b206884bdbe137ffc3636f8b6fa759601336d"}
Oct 30 13:21:07.729932 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:07.729782    4943 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5ec4624a6d71b069467a9c75410b206884bdbe137ffc3636f8b6fa759601336d"
Oct 30 13:21:08.535069 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:08.535027    4943 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-f462c87d-t6b8r"]
Oct 30 13:21:08.535775 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:08.535755    4943 cpu_manager.go:414] "containerMap: removing container" podUID="bd28f696-44c5-49ef-ac4b-9b63e9b7e5bb" containerName="revoke-keys"
Oct 30 13:21:08.535833 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:08.535776    4943 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="bd28f696-44c5-49ef-ac4b-9b63e9b7e5bb" containerName="revoke-keys"
Oct 30 13:21:08.535871 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:08.535866    4943 memory_manager.go:365] "RemoveStaleState removing state" podUID="bd28f696-44c5-49ef-ac4b-9b63e9b7e5bb" containerName="revoke-keys"
Oct 30 13:21:08.541152 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:08.541131    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f462c87d-t6b8r"
Oct 30 13:21:08.559771 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:08.559745    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-f462c87d-t6b8r"]
Oct 30 13:21:08.569480 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:08.569152    4943 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Oct 30 13:21:08.569480 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:08.569218    4943 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-p9z57\""
Oct 30 13:21:08.569480 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:08.569172    4943 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 13:21:08.569480 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:08.569170    4943 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Oct 30 13:21:08.657542 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:08.657513    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/2b3fefeb-d946-48e8-afab-773935b7a4e7-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-f462c87d-t6b8r\" (UID: \"2b3fefeb-d946-48e8-afab-773935b7a4e7\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f462c87d-t6b8r"
Oct 30 13:21:08.657677 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:08.657620    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mrqtn\" (UniqueName: \"kubernetes.io/projected/2b3fefeb-d946-48e8-afab-773935b7a4e7-kube-api-access-mrqtn\") pod \"maas-api-revoke-sub-f462c87d-t6b8r\" (UID: \"2b3fefeb-d946-48e8-afab-773935b7a4e7\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f462c87d-t6b8r"
Oct 30 13:21:08.657777 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:08.657761    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/2b3fefeb-d946-48e8-afab-773935b7a4e7-maas-api-xks-ca\") pod \"maas-api-revoke-sub-f462c87d-t6b8r\" (UID: \"2b3fefeb-d946-48e8-afab-773935b7a4e7\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f462c87d-t6b8r"
Oct 30 13:21:08.758482 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:08.758456    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/2b3fefeb-d946-48e8-afab-773935b7a4e7-maas-api-xks-ca\") pod \"maas-api-revoke-sub-f462c87d-t6b8r\" (UID: \"2b3fefeb-d946-48e8-afab-773935b7a4e7\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f462c87d-t6b8r"
Oct 30 13:21:08.758618 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:08.758515    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/2b3fefeb-d946-48e8-afab-773935b7a4e7-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-f462c87d-t6b8r\" (UID: \"2b3fefeb-d946-48e8-afab-773935b7a4e7\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f462c87d-t6b8r"
Oct 30 13:21:08.758618 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:08.758559    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mrqtn\" (UniqueName: \"kubernetes.io/projected/2b3fefeb-d946-48e8-afab-773935b7a4e7-kube-api-access-mrqtn\") pod \"maas-api-revoke-sub-f462c87d-t6b8r\" (UID: \"2b3fefeb-d946-48e8-afab-773935b7a4e7\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f462c87d-t6b8r"
Oct 30 13:21:08.759237 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:08.759211    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/2b3fefeb-d946-48e8-afab-773935b7a4e7-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-f462c87d-t6b8r\" (UID: \"2b3fefeb-d946-48e8-afab-773935b7a4e7\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f462c87d-t6b8r"
Oct 30 13:21:08.760849 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:08.760826    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/2b3fefeb-d946-48e8-afab-773935b7a4e7-maas-api-xks-ca\") pod \"maas-api-revoke-sub-f462c87d-t6b8r\" (UID: \"2b3fefeb-d946-48e8-afab-773935b7a4e7\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f462c87d-t6b8r"
Oct 30 13:21:08.789620 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:08.789560    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mrqtn\" (UniqueName: \"kubernetes.io/projected/2b3fefeb-d946-48e8-afab-773935b7a4e7-kube-api-access-mrqtn\") pod \"maas-api-revoke-sub-f462c87d-t6b8r\" (UID: \"2b3fefeb-d946-48e8-afab-773935b7a4e7\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f462c87d-t6b8r"
Oct 30 13:21:08.854722 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:08.854703    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f462c87d-t6b8r"
Oct 30 13:21:09.200800 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:09.200771    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-f462c87d-t6b8r"]
Oct 30 13:21:09.200923 ip-10-0-142-84 kubenswrapper[4943]: W1030 13:21:09.200836    4943 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2b3fefeb_d946_48e8_afab_773935b7a4e7.slice/crio-64fe38afbceb630a83e75c800479f548e828c336bc08136b900934362c769c11 WatchSource:0}: Error finding container 64fe38afbceb630a83e75c800479f548e828c336bc08136b900934362c769c11: Status 404 returned error can't find the container with id 64fe38afbceb630a83e75c800479f548e828c336bc08136b900934362c769c11
Oct 30 13:21:09.745001 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:09.744913    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f462c87d-t6b8r" event={"ID":"2b3fefeb-d946-48e8-afab-773935b7a4e7","Type":"ContainerStarted","Data":"e2a612caa8cab1756c976f5ec2da9e734210935ae608a8342fc803db964c2c7f"}
Oct 30 13:21:09.745001 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:09.744967    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f462c87d-t6b8r" event={"ID":"2b3fefeb-d946-48e8-afab-773935b7a4e7","Type":"ContainerStarted","Data":"64fe38afbceb630a83e75c800479f548e828c336bc08136b900934362c769c11"}
Oct 30 13:21:09.799346 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:09.799296    4943 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f462c87d-t6b8r" podStartSLOduration=1.799283245 podStartE2EDuration="1.799283245s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:21:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:21:09.797402899 +0000 UTC m=+1445.585406629" watchObservedRunningTime="2026-10-30 13:21:09.799283245 +0000 UTC m=+1445.587286975"
Oct 30 13:21:10.214771 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:10.214734    4943 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w4-88f6a4/distinct-2-w4-88f6a4-kserve-7b8f7bd7dc-5bt68" podUID="fe434953-1a27-4c4f-9478-f8d0264c8f61" containerName="main" containerID="cri-o://e4b545b668a00853f418cc56179cd7dc3276a4b4cb1f5368e68d2ee9f84ff492" gracePeriod=45
Oct 30 13:21:10.524194 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:10.524169    4943 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-88f6a4/distinct-2-w4-88f6a4-kserve-7b8f7bd7dc-5bt68"
Oct 30 13:21:10.678950 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:10.677768    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fe434953-1a27-4c4f-9478-f8d0264c8f61-home\" (UniqueName: \"kubernetes.io/empty-dir/fe434953-1a27-4c4f-9478-f8d0264c8f61-home\") pod \"fe434953-1a27-4c4f-9478-f8d0264c8f61\" (UID: \"fe434953-1a27-4c4f-9478-f8d0264c8f61\") "
Oct 30 13:21:10.678950 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:10.677856    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fe434953-1a27-4c4f-9478-f8d0264c8f61-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fe434953-1a27-4c4f-9478-f8d0264c8f61-tmp-dir\") pod \"fe434953-1a27-4c4f-9478-f8d0264c8f61\" (UID: \"fe434953-1a27-4c4f-9478-f8d0264c8f61\") "
Oct 30 13:21:10.678950 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:10.677922    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/fe434953-1a27-4c4f-9478-f8d0264c8f61-tls-certs\" (UniqueName: \"kubernetes.io/secret/fe434953-1a27-4c4f-9478-f8d0264c8f61-tls-certs\") pod \"fe434953-1a27-4c4f-9478-f8d0264c8f61\" (UID: \"fe434953-1a27-4c4f-9478-f8d0264c8f61\") "
Oct 30 13:21:10.678950 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:10.677974    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fe434953-1a27-4c4f-9478-f8d0264c8f61-dshm\" (UniqueName: \"kubernetes.io/empty-dir/fe434953-1a27-4c4f-9478-f8d0264c8f61-dshm\") pod \"fe434953-1a27-4c4f-9478-f8d0264c8f61\" (UID: \"fe434953-1a27-4c4f-9478-f8d0264c8f61\") "
Oct 30 13:21:10.678950 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:10.678144    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fe434953-1a27-4c4f-9478-f8d0264c8f61-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fe434953-1a27-4c4f-9478-f8d0264c8f61-model-cache\") pod \"fe434953-1a27-4c4f-9478-f8d0264c8f61\" (UID: \"fe434953-1a27-4c4f-9478-f8d0264c8f61\") "
Oct 30 13:21:10.678950 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:10.678179    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fe434953-1a27-4c4f-9478-f8d0264c8f61-kube-api-access-vgtqw\" (UniqueName: \"kubernetes.io/projected/fe434953-1a27-4c4f-9478-f8d0264c8f61-kube-api-access-vgtqw\") pod \"fe434953-1a27-4c4f-9478-f8d0264c8f61\" (UID: \"fe434953-1a27-4c4f-9478-f8d0264c8f61\") "
Oct 30 13:21:10.679701 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:10.679671    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fe434953-1a27-4c4f-9478-f8d0264c8f61-home" pod "fe434953-1a27-4c4f-9478-f8d0264c8f61" (UID: "fe434953-1a27-4c4f-9478-f8d0264c8f61"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:21:10.679918 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:10.679895    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fe434953-1a27-4c4f-9478-f8d0264c8f61-model-cache" pod "fe434953-1a27-4c4f-9478-f8d0264c8f61" (UID: "fe434953-1a27-4c4f-9478-f8d0264c8f61"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:21:10.680104 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:10.680082    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fe434953-1a27-4c4f-9478-f8d0264c8f61-tmp-dir" pod "fe434953-1a27-4c4f-9478-f8d0264c8f61" (UID: "fe434953-1a27-4c4f-9478-f8d0264c8f61"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:21:10.681830 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:10.681792    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fe434953-1a27-4c4f-9478-f8d0264c8f61-kube-api-access-vgtqw" pod "fe434953-1a27-4c4f-9478-f8d0264c8f61" (UID: "fe434953-1a27-4c4f-9478-f8d0264c8f61"). InnerVolumeSpecName "kube-api-access-vgtqw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:21:10.682173 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:10.682140    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fe434953-1a27-4c4f-9478-f8d0264c8f61-tls-certs" pod "fe434953-1a27-4c4f-9478-f8d0264c8f61" (UID: "fe434953-1a27-4c4f-9478-f8d0264c8f61"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 13:21:10.683427 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:10.683393    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fe434953-1a27-4c4f-9478-f8d0264c8f61-dshm" pod "fe434953-1a27-4c4f-9478-f8d0264c8f61" (UID: "fe434953-1a27-4c4f-9478-f8d0264c8f61"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:21:10.753762 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:10.753727    4943 generic.go:356] "Generic (PLEG): container finished" podID="fe434953-1a27-4c4f-9478-f8d0264c8f61" containerID="e4b545b668a00853f418cc56179cd7dc3276a4b4cb1f5368e68d2ee9f84ff492" exitCode=0
Oct 30 13:21:10.754207 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:10.753791    4943 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-88f6a4/distinct-2-w4-88f6a4-kserve-7b8f7bd7dc-5bt68"
Oct 30 13:21:10.754207 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:10.753816    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-88f6a4/distinct-2-w4-88f6a4-kserve-7b8f7bd7dc-5bt68" event={"ID":"fe434953-1a27-4c4f-9478-f8d0264c8f61","Type":"ContainerDied","Data":"e4b545b668a00853f418cc56179cd7dc3276a4b4cb1f5368e68d2ee9f84ff492"}
Oct 30 13:21:10.754207 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:10.753867    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-88f6a4/distinct-2-w4-88f6a4-kserve-7b8f7bd7dc-5bt68" event={"ID":"fe434953-1a27-4c4f-9478-f8d0264c8f61","Type":"ContainerDied","Data":"37bd20dc0155c4f8e7bf00c589a7888ef60f582897d64627c794dd0b34d75882"}
Oct 30 13:21:10.754207 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:10.753891    4943 scope.go:122] "RemoveContainer" containerID="e4b545b668a00853f418cc56179cd7dc3276a4b4cb1f5368e68d2ee9f84ff492"
Oct 30 13:21:10.755920 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:10.755898    4943 generic.go:356] "Generic (PLEG): container finished" podID="2b3fefeb-d946-48e8-afab-773935b7a4e7" containerID="e2a612caa8cab1756c976f5ec2da9e734210935ae608a8342fc803db964c2c7f" exitCode=0
Oct 30 13:21:10.756024 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:10.755972    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f462c87d-t6b8r" event={"ID":"2b3fefeb-d946-48e8-afab-773935b7a4e7","Type":"ContainerDied","Data":"e2a612caa8cab1756c976f5ec2da9e734210935ae608a8342fc803db964c2c7f"}
Oct 30 13:21:10.766441 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:10.766426    4943 scope.go:122] "RemoveContainer" containerID="e4b545b668a00853f418cc56179cd7dc3276a4b4cb1f5368e68d2ee9f84ff492"
Oct 30 13:21:10.766734 ip-10-0-142-84 kubenswrapper[4943]: E1030 13:21:10.766715    4943 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e4b545b668a00853f418cc56179cd7dc3276a4b4cb1f5368e68d2ee9f84ff492\": container with ID starting with e4b545b668a00853f418cc56179cd7dc3276a4b4cb1f5368e68d2ee9f84ff492 not found: ID does not exist" containerID="e4b545b668a00853f418cc56179cd7dc3276a4b4cb1f5368e68d2ee9f84ff492"
Oct 30 13:21:10.766793 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:10.766741    4943 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e4b545b668a00853f418cc56179cd7dc3276a4b4cb1f5368e68d2ee9f84ff492"} err="failed to get container status \"e4b545b668a00853f418cc56179cd7dc3276a4b4cb1f5368e68d2ee9f84ff492\": rpc error: code = NotFound desc = could not find container \"e4b545b668a00853f418cc56179cd7dc3276a4b4cb1f5368e68d2ee9f84ff492\": container with ID starting with e4b545b668a00853f418cc56179cd7dc3276a4b4cb1f5368e68d2ee9f84ff492 not found: ID does not exist"
Oct 30 13:21:10.779738 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:10.779718    4943 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fe434953-1a27-4c4f-9478-f8d0264c8f61-model-cache\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:21:10.779812 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:10.779739    4943 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vgtqw\" (UniqueName: \"kubernetes.io/projected/fe434953-1a27-4c4f-9478-f8d0264c8f61-kube-api-access-vgtqw\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:21:10.779812 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:10.779749    4943 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fe434953-1a27-4c4f-9478-f8d0264c8f61-home\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:21:10.779812 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:10.779760    4943 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fe434953-1a27-4c4f-9478-f8d0264c8f61-tmp-dir\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:21:10.779812 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:10.779768    4943 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fe434953-1a27-4c4f-9478-f8d0264c8f61-tls-certs\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:21:10.779812 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:10.779776    4943 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fe434953-1a27-4c4f-9478-f8d0264c8f61-dshm\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:21:10.858038 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:10.857996    4943 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-88f6a4/distinct-2-w4-88f6a4-kserve-7b8f7bd7dc-5bt68"]
Oct 30 13:21:10.862892 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:10.862869    4943 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w4-88f6a4/distinct-2-w4-88f6a4-kserve-7b8f7bd7dc-5bt68"]
Oct 30 13:21:11.920820 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:11.920784    4943 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f462c87d-t6b8r"
Oct 30 13:21:11.990190 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:11.990152    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/2b3fefeb-d946-48e8-afab-773935b7a4e7-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/2b3fefeb-d946-48e8-afab-773935b7a4e7-maas-api-xks-ca\") pod \"2b3fefeb-d946-48e8-afab-773935b7a4e7\" (UID: \"2b3fefeb-d946-48e8-afab-773935b7a4e7\") "
Oct 30 13:21:11.990556 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:11.990529    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2b3fefeb-d946-48e8-afab-773935b7a4e7-kube-api-access-mrqtn\" (UniqueName: \"kubernetes.io/projected/2b3fefeb-d946-48e8-afab-773935b7a4e7-kube-api-access-mrqtn\") pod \"2b3fefeb-d946-48e8-afab-773935b7a4e7\" (UID: \"2b3fefeb-d946-48e8-afab-773935b7a4e7\") "
Oct 30 13:21:11.990677 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:11.990589    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/2b3fefeb-d946-48e8-afab-773935b7a4e7-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/2b3fefeb-d946-48e8-afab-773935b7a4e7-maas-api-openshift-ca\") pod \"2b3fefeb-d946-48e8-afab-773935b7a4e7\" (UID: \"2b3fefeb-d946-48e8-afab-773935b7a4e7\") "
Oct 30 13:21:11.991342 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:11.991316    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2b3fefeb-d946-48e8-afab-773935b7a4e7-maas-api-openshift-ca" pod "2b3fefeb-d946-48e8-afab-773935b7a4e7" (UID: "2b3fefeb-d946-48e8-afab-773935b7a4e7"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 13:21:11.994212 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:11.994185    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2b3fefeb-d946-48e8-afab-773935b7a4e7-kube-api-access-mrqtn" pod "2b3fefeb-d946-48e8-afab-773935b7a4e7" (UID: "2b3fefeb-d946-48e8-afab-773935b7a4e7"). InnerVolumeSpecName "kube-api-access-mrqtn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:21:11.998418 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:11.998389    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2b3fefeb-d946-48e8-afab-773935b7a4e7-maas-api-xks-ca" pod "2b3fefeb-d946-48e8-afab-773935b7a4e7" (UID: "2b3fefeb-d946-48e8-afab-773935b7a4e7"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 13:21:12.098434 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:12.098283    4943 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/2b3fefeb-d946-48e8-afab-773935b7a4e7-maas-api-xks-ca\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:21:12.098434 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:12.098327    4943 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mrqtn\" (UniqueName: \"kubernetes.io/projected/2b3fefeb-d946-48e8-afab-773935b7a4e7-kube-api-access-mrqtn\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:21:12.098434 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:12.098344    4943 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/2b3fefeb-d946-48e8-afab-773935b7a4e7-maas-api-openshift-ca\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:21:12.350423 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:12.350341    4943 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="fe434953-1a27-4c4f-9478-f8d0264c8f61" path="/var/lib/kubelet/pods/fe434953-1a27-4c4f-9478-f8d0264c8f61/volumes"
Oct 30 13:21:12.365096 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:12.365070    4943 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w2-f76c09-mhrhl"]
Oct 30 13:21:12.368862 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:12.368831    4943 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w2-f76c09-mhrhl"]
Oct 30 13:21:12.378329 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:12.378308    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-hdmrm"]
Oct 30 13:21:12.772509 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:12.772479    4943 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f462c87d-t6b8r"
Oct 30 13:21:12.772708 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:12.772508    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f462c87d-t6b8r" event={"ID":"2b3fefeb-d946-48e8-afab-773935b7a4e7","Type":"ContainerDied","Data":"64fe38afbceb630a83e75c800479f548e828c336bc08136b900934362c769c11"}
Oct 30 13:21:12.772708 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:12.772561    4943 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="64fe38afbceb630a83e75c800479f548e828c336bc08136b900934362c769c11"
Oct 30 13:21:14.357031 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:14.356991    4943 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b3dce7a0-6891-4b62-86bd-12cb7c93f94c" path="/var/lib/kubelet/pods/b3dce7a0-6891-4b62-86bd-12cb7c93f94c/volumes"
Oct 30 13:21:16.070952 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:16.070245    4943 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w4-88f6a4-7dd558b944-gkzck" podUID="80c84ecd-0a11-4d8e-ab46-2ba9efd4e676" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 13:21:16.245844 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:16.245814    4943 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-88f6a4-7tfvz"]
Oct 30 13:21:16.259700 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:16.259673    4943 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-88f6a4-7tfvz"]
Oct 30 13:21:16.383105 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:16.383062    4943 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5b325a80-09c9-4350-8e01-f50cfab3e2a7" path="/var/lib/kubelet/pods/5b325a80-09c9-4350-8e01-f50cfab3e2a7/volumes"
Oct 30 13:21:16.537695 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:16.537669    4943 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w2-f76c09-588d998d5b-z94mv"
Oct 30 13:21:16.573642 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:16.573616    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/278a0939-7f8a-4551-bac6-a4ef6b5f44a9-kube-api-access-5wd78\" (UniqueName: \"kubernetes.io/projected/278a0939-7f8a-4551-bac6-a4ef6b5f44a9-kube-api-access-5wd78\") pod \"278a0939-7f8a-4551-bac6-a4ef6b5f44a9\" (UID: \"278a0939-7f8a-4551-bac6-a4ef6b5f44a9\") "
Oct 30 13:21:16.573755 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:16.573698    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/278a0939-7f8a-4551-bac6-a4ef6b5f44a9-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/278a0939-7f8a-4551-bac6-a4ef6b5f44a9-plugins-config-volume\") pod \"278a0939-7f8a-4551-bac6-a4ef6b5f44a9\" (UID: \"278a0939-7f8a-4551-bac6-a4ef6b5f44a9\") "
Oct 30 13:21:16.574072 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:16.574048    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/278a0939-7f8a-4551-bac6-a4ef6b5f44a9-plugins-config-volume" pod "278a0939-7f8a-4551-bac6-a4ef6b5f44a9" (UID: "278a0939-7f8a-4551-bac6-a4ef6b5f44a9"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 13:21:16.575973 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:16.575952    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/278a0939-7f8a-4551-bac6-a4ef6b5f44a9-kube-api-access-5wd78" pod "278a0939-7f8a-4551-bac6-a4ef6b5f44a9" (UID: "278a0939-7f8a-4551-bac6-a4ef6b5f44a9"). InnerVolumeSpecName "kube-api-access-5wd78". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:21:16.578111 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:16.578089    4943 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-f76c09-5bdd4bf999-gbp8q"
Oct 30 13:21:16.675290 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:16.675190    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f35ce2f9-b769-4ec4-9c57-a31057339d93-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f35ce2f9-b769-4ec4-9c57-a31057339d93-plugins-config-volume\") pod \"f35ce2f9-b769-4ec4-9c57-a31057339d93\" (UID: \"f35ce2f9-b769-4ec4-9c57-a31057339d93\") "
Oct 30 13:21:16.675290 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:16.675282    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f35ce2f9-b769-4ec4-9c57-a31057339d93-kube-api-access-d6rm7\" (UniqueName: \"kubernetes.io/projected/f35ce2f9-b769-4ec4-9c57-a31057339d93-kube-api-access-d6rm7\") pod \"f35ce2f9-b769-4ec4-9c57-a31057339d93\" (UID: \"f35ce2f9-b769-4ec4-9c57-a31057339d93\") "
Oct 30 13:21:16.675653 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:16.675627    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f35ce2f9-b769-4ec4-9c57-a31057339d93-plugins-config-volume" pod "f35ce2f9-b769-4ec4-9c57-a31057339d93" (UID: "f35ce2f9-b769-4ec4-9c57-a31057339d93"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 13:21:16.675743 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:16.675693    4943 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5wd78\" (UniqueName: \"kubernetes.io/projected/278a0939-7f8a-4551-bac6-a4ef6b5f44a9-kube-api-access-5wd78\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:21:16.675743 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:16.675709    4943 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/278a0939-7f8a-4551-bac6-a4ef6b5f44a9-plugins-config-volume\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:21:16.677380 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:16.677353    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f35ce2f9-b769-4ec4-9c57-a31057339d93-kube-api-access-d6rm7" pod "f35ce2f9-b769-4ec4-9c57-a31057339d93" (UID: "f35ce2f9-b769-4ec4-9c57-a31057339d93"). InnerVolumeSpecName "kube-api-access-d6rm7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:21:16.777268 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:16.777226    4943 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f35ce2f9-b769-4ec4-9c57-a31057339d93-plugins-config-volume\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:21:16.777431 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:16.777255    4943 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d6rm7\" (UniqueName: \"kubernetes.io/projected/f35ce2f9-b769-4ec4-9c57-a31057339d93-kube-api-access-d6rm7\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:21:16.834156 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:16.834117    4943 generic.go:356] "Generic (PLEG): container finished" podID="278a0939-7f8a-4551-bac6-a4ef6b5f44a9" containerID="961bb93619aa61ec76505cd093319d4b9f87decb2de092205a0d403e678f5366" exitCode=0
Oct 30 13:21:16.834308 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:16.834164    4943 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w2-f76c09-588d998d5b-z94mv"
Oct 30 13:21:16.834308 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:16.834208    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w2-f76c09-588d998d5b-z94mv" event={"ID":"278a0939-7f8a-4551-bac6-a4ef6b5f44a9","Type":"ContainerDied","Data":"961bb93619aa61ec76505cd093319d4b9f87decb2de092205a0d403e678f5366"}
Oct 30 13:21:16.834308 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:16.834269    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w2-f76c09-588d998d5b-z94mv" event={"ID":"278a0939-7f8a-4551-bac6-a4ef6b5f44a9","Type":"ContainerDied","Data":"d739aa38c7b9121bc9775ee4aa10cb488e5faae26be150f58c21a1fdb0ea6429"}
Oct 30 13:21:16.834308 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:16.834297    4943 scope.go:122] "RemoveContainer" containerID="961bb93619aa61ec76505cd093319d4b9f87decb2de092205a0d403e678f5366"
Oct 30 13:21:16.835761 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:16.835737    4943 generic.go:356] "Generic (PLEG): container finished" podID="f35ce2f9-b769-4ec4-9c57-a31057339d93" containerID="74da8e5e76cf9d27e72eb45e48ad35c66ed936f0c10713c1c015d00666116b29" exitCode=0
Oct 30 13:21:16.835855 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:16.835802    4943 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-f76c09-5bdd4bf999-gbp8q"
Oct 30 13:21:16.835855 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:16.835823    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-f76c09-5bdd4bf999-gbp8q" event={"ID":"f35ce2f9-b769-4ec4-9c57-a31057339d93","Type":"ContainerDied","Data":"74da8e5e76cf9d27e72eb45e48ad35c66ed936f0c10713c1c015d00666116b29"}
Oct 30 13:21:16.835948 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:16.835871    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-f76c09-5bdd4bf999-gbp8q" event={"ID":"f35ce2f9-b769-4ec4-9c57-a31057339d93","Type":"ContainerDied","Data":"d7e07a217a15b26e7ab1e5db8e20d35dd7cb51a93197af04904d5dfce1bf0221"}
Oct 30 13:21:16.845382 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:16.845335    4943 scope.go:122] "RemoveContainer" containerID="961bb93619aa61ec76505cd093319d4b9f87decb2de092205a0d403e678f5366"
Oct 30 13:21:16.845703 ip-10-0-142-84 kubenswrapper[4943]: E1030 13:21:16.845677    4943 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"961bb93619aa61ec76505cd093319d4b9f87decb2de092205a0d403e678f5366\": container with ID starting with 961bb93619aa61ec76505cd093319d4b9f87decb2de092205a0d403e678f5366 not found: ID does not exist" containerID="961bb93619aa61ec76505cd093319d4b9f87decb2de092205a0d403e678f5366"
Oct 30 13:21:16.845795 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:16.845711    4943 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"961bb93619aa61ec76505cd093319d4b9f87decb2de092205a0d403e678f5366"} err="failed to get container status \"961bb93619aa61ec76505cd093319d4b9f87decb2de092205a0d403e678f5366\": rpc error: code = NotFound desc = could not find container \"961bb93619aa61ec76505cd093319d4b9f87decb2de092205a0d403e678f5366\": container with ID starting with 961bb93619aa61ec76505cd093319d4b9f87decb2de092205a0d403e678f5366 not found: ID does not exist"
Oct 30 13:21:16.845795 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:16.845730    4943 scope.go:122] "RemoveContainer" containerID="74da8e5e76cf9d27e72eb45e48ad35c66ed936f0c10713c1c015d00666116b29"
Oct 30 13:21:16.855327 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:16.855309    4943 scope.go:122] "RemoveContainer" containerID="74da8e5e76cf9d27e72eb45e48ad35c66ed936f0c10713c1c015d00666116b29"
Oct 30 13:21:16.855579 ip-10-0-142-84 kubenswrapper[4943]: E1030 13:21:16.855559    4943 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"74da8e5e76cf9d27e72eb45e48ad35c66ed936f0c10713c1c015d00666116b29\": container with ID starting with 74da8e5e76cf9d27e72eb45e48ad35c66ed936f0c10713c1c015d00666116b29 not found: ID does not exist" containerID="74da8e5e76cf9d27e72eb45e48ad35c66ed936f0c10713c1c015d00666116b29"
Oct 30 13:21:16.855645 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:16.855585    4943 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"74da8e5e76cf9d27e72eb45e48ad35c66ed936f0c10713c1c015d00666116b29"} err="failed to get container status \"74da8e5e76cf9d27e72eb45e48ad35c66ed936f0c10713c1c015d00666116b29\": rpc error: code = NotFound desc = could not find container \"74da8e5e76cf9d27e72eb45e48ad35c66ed936f0c10713c1c015d00666116b29\": container with ID starting with 74da8e5e76cf9d27e72eb45e48ad35c66ed936f0c10713c1c015d00666116b29 not found: ID does not exist"
Oct 30 13:21:16.899952 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:16.899919    4943 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w2-f76c09-5bdd4bf999-gbp8q"]
Oct 30 13:21:16.903556 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:16.903534    4943 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w2-f76c09-5bdd4bf999-gbp8q"]
Oct 30 13:21:16.956147 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:16.956123    4943 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w2-f76c09-588d998d5b-z94mv"]
Oct 30 13:21:16.963068 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:16.963044    4943 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w2-f76c09-588d998d5b-z94mv"]
Oct 30 13:21:18.350574 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:18.350541    4943 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="278a0939-7f8a-4551-bac6-a4ef6b5f44a9" path="/var/lib/kubelet/pods/278a0939-7f8a-4551-bac6-a4ef6b5f44a9/volumes"
Oct 30 13:21:18.350958 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:18.350929    4943 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f35ce2f9-b769-4ec4-9c57-a31057339d93" path="/var/lib/kubelet/pods/f35ce2f9-b769-4ec4-9c57-a31057339d93/volumes"
Oct 30 13:21:19.667857 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:19.667816    4943 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w6-ab3176-2fdzj"]
Oct 30 13:21:19.668587 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:19.668563    4943 cpu_manager.go:414] "containerMap: removing container" podUID="f35ce2f9-b769-4ec4-9c57-a31057339d93" containerName="payload-pre-processing"
Oct 30 13:21:19.668697 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:19.668587    4943 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f35ce2f9-b769-4ec4-9c57-a31057339d93" containerName="payload-pre-processing"
Oct 30 13:21:19.668697 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:19.668601    4943 cpu_manager.go:414] "containerMap: removing container" podUID="278a0939-7f8a-4551-bac6-a4ef6b5f44a9" containerName="payload-processing"
Oct 30 13:21:19.668697 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:19.668610    4943 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="278a0939-7f8a-4551-bac6-a4ef6b5f44a9" containerName="payload-processing"
Oct 30 13:21:19.668697 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:19.668622    4943 cpu_manager.go:414] "containerMap: removing container" podUID="fe434953-1a27-4c4f-9478-f8d0264c8f61" containerName="main"
Oct 30 13:21:19.668697 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:19.668629    4943 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fe434953-1a27-4c4f-9478-f8d0264c8f61" containerName="main"
Oct 30 13:21:19.668697 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:19.668648    4943 cpu_manager.go:414] "containerMap: removing container" podUID="2b3fefeb-d946-48e8-afab-773935b7a4e7" containerName="revoke-keys"
Oct 30 13:21:19.668697 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:19.668656    4943 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2b3fefeb-d946-48e8-afab-773935b7a4e7" containerName="revoke-keys"
Oct 30 13:21:19.669034 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:19.668815    4943 memory_manager.go:365] "RemoveStaleState removing state" podUID="f35ce2f9-b769-4ec4-9c57-a31057339d93" containerName="payload-pre-processing"
Oct 30 13:21:19.669034 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:19.668826    4943 memory_manager.go:365] "RemoveStaleState removing state" podUID="278a0939-7f8a-4551-bac6-a4ef6b5f44a9" containerName="payload-processing"
Oct 30 13:21:19.669034 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:19.668838    4943 memory_manager.go:365] "RemoveStaleState removing state" podUID="fe434953-1a27-4c4f-9478-f8d0264c8f61" containerName="main"
Oct 30 13:21:19.669034 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:19.668850    4943 memory_manager.go:365] "RemoveStaleState removing state" podUID="2b3fefeb-d946-48e8-afab-773935b7a4e7" containerName="revoke-keys"
Oct 30 13:21:19.673543 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:19.673511    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w6-ab3176-2fdzj"
Oct 30 13:21:19.713339 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:19.707685    4943 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Oct 30 13:21:19.713339 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:19.707885    4943 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Oct 30 13:21:19.713339 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:19.708701    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/87aca420-1995-4b47-a522-0d60b607db07-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-worker-w6-ab3176-2fdzj\" (UID: \"87aca420-1995-4b47-a522-0d60b607db07\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w6-ab3176-2fdzj"
Oct 30 13:21:19.713339 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:19.709033    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lx9xz\" (UniqueName: \"kubernetes.io/projected/87aca420-1995-4b47-a522-0d60b607db07-kube-api-access-lx9xz\") pod \"maas-api-revoke-keys-e2e-worker-w6-ab3176-2fdzj\" (UID: \"87aca420-1995-4b47-a522-0d60b607db07\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w6-ab3176-2fdzj"
Oct 30 13:21:19.713339 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:19.709128    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/87aca420-1995-4b47-a522-0d60b607db07-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-worker-w6-ab3176-2fdzj\" (UID: \"87aca420-1995-4b47-a522-0d60b607db07\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w6-ab3176-2fdzj"
Oct 30 13:21:19.732029 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:19.731761    4943 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-p9z57\""
Oct 30 13:21:19.732029 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:19.731832    4943 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 13:21:19.778757 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:19.778713    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w6-ab3176-2fdzj"]
Oct 30 13:21:19.828323 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:19.828285    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/87aca420-1995-4b47-a522-0d60b607db07-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-worker-w6-ab3176-2fdzj\" (UID: \"87aca420-1995-4b47-a522-0d60b607db07\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w6-ab3176-2fdzj"
Oct 30 13:21:19.828521 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:19.828398    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/87aca420-1995-4b47-a522-0d60b607db07-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-worker-w6-ab3176-2fdzj\" (UID: \"87aca420-1995-4b47-a522-0d60b607db07\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w6-ab3176-2fdzj"
Oct 30 13:21:19.828521 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:19.828465    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lx9xz\" (UniqueName: \"kubernetes.io/projected/87aca420-1995-4b47-a522-0d60b607db07-kube-api-access-lx9xz\") pod \"maas-api-revoke-keys-e2e-worker-w6-ab3176-2fdzj\" (UID: \"87aca420-1995-4b47-a522-0d60b607db07\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w6-ab3176-2fdzj"
Oct 30 13:21:19.828949 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:19.828927    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/87aca420-1995-4b47-a522-0d60b607db07-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-worker-w6-ab3176-2fdzj\" (UID: \"87aca420-1995-4b47-a522-0d60b607db07\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w6-ab3176-2fdzj"
Oct 30 13:21:19.830798 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:19.830773    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/87aca420-1995-4b47-a522-0d60b607db07-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-worker-w6-ab3176-2fdzj\" (UID: \"87aca420-1995-4b47-a522-0d60b607db07\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w6-ab3176-2fdzj"
Oct 30 13:21:19.854219 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:19.854188    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lx9xz\" (UniqueName: \"kubernetes.io/projected/87aca420-1995-4b47-a522-0d60b607db07-kube-api-access-lx9xz\") pod \"maas-api-revoke-keys-e2e-worker-w6-ab3176-2fdzj\" (UID: \"87aca420-1995-4b47-a522-0d60b607db07\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w6-ab3176-2fdzj"
Oct 30 13:21:20.046280 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:20.042527    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w6-ab3176-2fdzj"
Oct 30 13:21:20.382560 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:20.382519    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w6-ab3176-2fdzj"]
Oct 30 13:21:20.383096 ip-10-0-142-84 kubenswrapper[4943]: W1030 13:21:20.383070    4943 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod87aca420_1995_4b47_a522_0d60b607db07.slice/crio-035b14b9946372d356b75a1d138cb1f7128a0490a79d74c513419a905788f9a6 WatchSource:0}: Error finding container 035b14b9946372d356b75a1d138cb1f7128a0490a79d74c513419a905788f9a6: Status 404 returned error can't find the container with id 035b14b9946372d356b75a1d138cb1f7128a0490a79d74c513419a905788f9a6
Oct 30 13:21:20.848337 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:20.848306    4943 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-mlh2m"]
Oct 30 13:21:20.848751 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:20.848504    4943 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-mlh2m" podUID="1c74e310-6638-4b15-97d8-65854946086a" containerName="istio-proxy" containerID="cri-o://343c2ddb8a87c1a0d9493df064e5f7fb86fe37d31cd4e3664dfeadb6b3cc133b" gracePeriod=30
Oct 30 13:21:20.868196 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:20.868161    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w6-ab3176-2fdzj" event={"ID":"87aca420-1995-4b47-a522-0d60b607db07","Type":"ContainerStarted","Data":"2acce7af2aba9816cfc70604ce6451017e0fca1506f5cdb6c9e2ccdbfb469c5f"}
Oct 30 13:21:20.868356 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:20.868222    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w6-ab3176-2fdzj" event={"ID":"87aca420-1995-4b47-a522-0d60b607db07","Type":"ContainerStarted","Data":"035b14b9946372d356b75a1d138cb1f7128a0490a79d74c513419a905788f9a6"}
Oct 30 13:21:21.006617 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:21.006584    4943 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-pph7x"]
Oct 30 13:21:21.006854 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:21.006828    4943 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-pph7x" podUID="bd83c13a-8f22-4ada-a894-90cd49e87782" containerName="istio-proxy" containerID="cri-o://c3864650b233dda9248352e7bb7d5bccbe2291f3806423c81a128038986a9903" gracePeriod=30
Oct 30 13:21:21.007543 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:21.007490    4943 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w6-ab3176-2fdzj" podStartSLOduration=2.007472987 podStartE2EDuration="2.007472987s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:21:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:21:20.999705948 +0000 UTC m=+1456.787709678" watchObservedRunningTime="2026-10-30 13:21:21.007472987 +0000 UTC m=+1456.795476721"
Oct 30 13:21:21.018227 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:21.018198    4943 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-ljkqn"]
Oct 30 13:21:21.018489 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:21.018443    4943 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-ljkqn" podUID="f0ab36b0-4ba8-41da-8fc2-6a70d1082599" containerName="istio-proxy" containerID="cri-o://3bb47869cf566fc54232cdffa43db4a7042cefe5994382c6cec1269971649d61" gracePeriod=30
Oct 30 13:21:21.028793 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:21.028767    4943 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-bf7pz"]
Oct 30 13:21:21.028985 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:21.028965    4943 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-bf7pz" podUID="258db5e7-aa22-4bb6-b8ee-844241ca5399" containerName="istio-proxy" containerID="cri-o://9989ec3323990145f3437b91850185cbffab2c42b9e6f132674e969ffb52b858" gracePeriod=30
Oct 30 13:21:21.032364 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:21.032342    4943 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-kpvbd"]
Oct 30 13:21:21.032539 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:21.032521    4943 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-kpvbd" podUID="f0e82596-0383-4ec9-a856-a6234e9da808" containerName="istio-proxy" containerID="cri-o://0717bcb2ccdab96dd7a6937fd08a0fe5581f31152a2d124a5306480719d7c3f7" gracePeriod=30
Oct 30 13:21:21.876581 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:21.876544    4943 generic.go:356] "Generic (PLEG): container finished" podID="87aca420-1995-4b47-a522-0d60b607db07" containerID="2acce7af2aba9816cfc70604ce6451017e0fca1506f5cdb6c9e2ccdbfb469c5f" exitCode=0
Oct 30 13:21:21.877008 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:21.876632    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w6-ab3176-2fdzj" event={"ID":"87aca420-1995-4b47-a522-0d60b607db07","Type":"ContainerDied","Data":"2acce7af2aba9816cfc70604ce6451017e0fca1506f5cdb6c9e2ccdbfb469c5f"}
Oct 30 13:21:22.970800 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:22.970441    4943 patch_prober.go:29] interesting pod/e2e-worker-w2-f76c09-openshift-default-65d5845b67-bf7pz container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 13:21:22.970800 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:22.970509    4943 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-bf7pz" podUID="258db5e7-aa22-4bb6-b8ee-844241ca5399" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 13:21:23.062938 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:23.062912    4943 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w6-ab3176-2fdzj"
Oct 30 13:21:23.181905 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:23.181872    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/87aca420-1995-4b47-a522-0d60b607db07-kube-api-access-lx9xz\" (UniqueName: \"kubernetes.io/projected/87aca420-1995-4b47-a522-0d60b607db07-kube-api-access-lx9xz\") pod \"87aca420-1995-4b47-a522-0d60b607db07\" (UID: \"87aca420-1995-4b47-a522-0d60b607db07\") "
Oct 30 13:21:23.182168 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:23.181958    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/87aca420-1995-4b47-a522-0d60b607db07-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/87aca420-1995-4b47-a522-0d60b607db07-maas-api-openshift-ca\") pod \"87aca420-1995-4b47-a522-0d60b607db07\" (UID: \"87aca420-1995-4b47-a522-0d60b607db07\") "
Oct 30 13:21:23.182168 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:23.182150    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/87aca420-1995-4b47-a522-0d60b607db07-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/87aca420-1995-4b47-a522-0d60b607db07-maas-api-xks-ca\") pod \"87aca420-1995-4b47-a522-0d60b607db07\" (UID: \"87aca420-1995-4b47-a522-0d60b607db07\") "
Oct 30 13:21:23.182358 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:23.182334    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/87aca420-1995-4b47-a522-0d60b607db07-maas-api-openshift-ca" pod "87aca420-1995-4b47-a522-0d60b607db07" (UID: "87aca420-1995-4b47-a522-0d60b607db07"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 13:21:23.182598 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:23.182578    4943 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/87aca420-1995-4b47-a522-0d60b607db07-maas-api-openshift-ca\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:21:23.184051 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:23.184024    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/87aca420-1995-4b47-a522-0d60b607db07-kube-api-access-lx9xz" pod "87aca420-1995-4b47-a522-0d60b607db07" (UID: "87aca420-1995-4b47-a522-0d60b607db07"). InnerVolumeSpecName "kube-api-access-lx9xz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:21:23.184137 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:23.184067    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/87aca420-1995-4b47-a522-0d60b607db07-maas-api-xks-ca" pod "87aca420-1995-4b47-a522-0d60b607db07" (UID: "87aca420-1995-4b47-a522-0d60b607db07"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 13:21:23.283960 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:23.283932    4943 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/87aca420-1995-4b47-a522-0d60b607db07-maas-api-xks-ca\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:21:23.283960 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:23.283958    4943 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lx9xz\" (UniqueName: \"kubernetes.io/projected/87aca420-1995-4b47-a522-0d60b607db07-kube-api-access-lx9xz\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:21:23.893420 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:23.893382    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w6-ab3176-2fdzj" event={"ID":"87aca420-1995-4b47-a522-0d60b607db07","Type":"ContainerDied","Data":"035b14b9946372d356b75a1d138cb1f7128a0490a79d74c513419a905788f9a6"}
Oct 30 13:21:23.893592 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:23.893438    4943 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="035b14b9946372d356b75a1d138cb1f7128a0490a79d74c513419a905788f9a6"
Oct 30 13:21:23.893592 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:23.893402    4943 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w6-ab3176-2fdzj"
Oct 30 13:21:24.452700 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:24.452667    4943 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w6-ab3176/simulator-w6-ab3176-kserve-6d9b64549f-dc8xz"]
Oct 30 13:21:25.578715 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:25.578675    4943 patch_prober.go:29] interesting pod/e2e-worker-w2-f76c09-openshift-default-65d5845b67-kpvbd container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 13:21:25.579094 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:25.578736    4943 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-kpvbd" podUID="f0e82596-0383-4ec9-a856-a6234e9da808" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 13:21:25.984020 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:25.983987    4943 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w4-88f6a4-7dd558b944-gkzck" podUID="80c84ecd-0a11-4d8e-ab46-2ba9efd4e676" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 13:21:26.315652 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:26.313692    4943 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-mlh2m"
Oct 30 13:21:26.334564 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:26.334536    4943 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-pph7x"
Oct 30 13:21:26.375349 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:26.375321    4943 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-bf7pz"
Oct 30 13:21:26.419458 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:26.419375    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1c74e310-6638-4b15-97d8-65854946086a-kube-api-access-5clq5\" (UniqueName: \"kubernetes.io/projected/1c74e310-6638-4b15-97d8-65854946086a-kube-api-access-5clq5\") pod \"1c74e310-6638-4b15-97d8-65854946086a\" (UID: \"1c74e310-6638-4b15-97d8-65854946086a\") "
Oct 30 13:21:26.419458 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:26.419419    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/258db5e7-aa22-4bb6-b8ee-844241ca5399-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/258db5e7-aa22-4bb6-b8ee-844241ca5399-istiod-ca-cert\") pod \"258db5e7-aa22-4bb6-b8ee-844241ca5399\" (UID: \"258db5e7-aa22-4bb6-b8ee-844241ca5399\") "
Oct 30 13:21:26.419767 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:26.419467    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/258db5e7-aa22-4bb6-b8ee-844241ca5399-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/258db5e7-aa22-4bb6-b8ee-844241ca5399-workload-certs\") pod \"258db5e7-aa22-4bb6-b8ee-844241ca5399\" (UID: \"258db5e7-aa22-4bb6-b8ee-844241ca5399\") "
Oct 30 13:21:26.419834 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:26.419762    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/258db5e7-aa22-4bb6-b8ee-844241ca5399-workload-certs" pod "258db5e7-aa22-4bb6-b8ee-844241ca5399" (UID: "258db5e7-aa22-4bb6-b8ee-844241ca5399"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:21:26.419834 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:26.419770    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/1c74e310-6638-4b15-97d8-65854946086a-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1c74e310-6638-4b15-97d8-65854946086a-istio-podinfo\") pod \"1c74e310-6638-4b15-97d8-65854946086a\" (UID: \"1c74e310-6638-4b15-97d8-65854946086a\") "
Oct 30 13:21:26.419834 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:26.419824    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bd83c13a-8f22-4ada-a894-90cd49e87782-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bd83c13a-8f22-4ada-a894-90cd49e87782-workload-socket\") pod \"bd83c13a-8f22-4ada-a894-90cd49e87782\" (UID: \"bd83c13a-8f22-4ada-a894-90cd49e87782\") "
Oct 30 13:21:26.420004 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:26.419870    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/258db5e7-aa22-4bb6-b8ee-844241ca5399-istio-token\" (UniqueName: \"kubernetes.io/projected/258db5e7-aa22-4bb6-b8ee-844241ca5399-istio-token\") pod \"258db5e7-aa22-4bb6-b8ee-844241ca5399\" (UID: \"258db5e7-aa22-4bb6-b8ee-844241ca5399\") "
Oct 30 13:21:26.420004 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:26.419902    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1c74e310-6638-4b15-97d8-65854946086a-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1c74e310-6638-4b15-97d8-65854946086a-workload-socket\") pod \"1c74e310-6638-4b15-97d8-65854946086a\" (UID: \"1c74e310-6638-4b15-97d8-65854946086a\") "
Oct 30 13:21:26.420004 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:26.419932    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/bd83c13a-8f22-4ada-a894-90cd49e87782-kube-api-access-xlprp\" (UniqueName: \"kubernetes.io/projected/bd83c13a-8f22-4ada-a894-90cd49e87782-kube-api-access-xlprp\") pod \"bd83c13a-8f22-4ada-a894-90cd49e87782\" (UID: \"bd83c13a-8f22-4ada-a894-90cd49e87782\") "
Oct 30 13:21:26.420004 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:26.419970    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1c74e310-6638-4b15-97d8-65854946086a-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1c74e310-6638-4b15-97d8-65854946086a-istio-envoy\") pod \"1c74e310-6638-4b15-97d8-65854946086a\" (UID: \"1c74e310-6638-4b15-97d8-65854946086a\") "
Oct 30 13:21:26.420220 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:26.420082    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1c74e310-6638-4b15-97d8-65854946086a-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1c74e310-6638-4b15-97d8-65854946086a-istiod-ca-cert\") pod \"1c74e310-6638-4b15-97d8-65854946086a\" (UID: \"1c74e310-6638-4b15-97d8-65854946086a\") "
Oct 30 13:21:26.420220 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:26.420113    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1c74e310-6638-4b15-97d8-65854946086a-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1c74e310-6638-4b15-97d8-65854946086a-istio-data\") pod \"1c74e310-6638-4b15-97d8-65854946086a\" (UID: \"1c74e310-6638-4b15-97d8-65854946086a\") "
Oct 30 13:21:26.420220 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:26.420143    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/bd83c13a-8f22-4ada-a894-90cd49e87782-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bd83c13a-8f22-4ada-a894-90cd49e87782-istio-podinfo\") pod \"bd83c13a-8f22-4ada-a894-90cd49e87782\" (UID: \"bd83c13a-8f22-4ada-a894-90cd49e87782\") "
Oct 30 13:21:26.420220 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:26.420180    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bd83c13a-8f22-4ada-a894-90cd49e87782-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bd83c13a-8f22-4ada-a894-90cd49e87782-credential-socket\") pod \"bd83c13a-8f22-4ada-a894-90cd49e87782\" (UID: \"bd83c13a-8f22-4ada-a894-90cd49e87782\") "
Oct 30 13:21:26.420457 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:26.420225    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1c74e310-6638-4b15-97d8-65854946086a-workload-socket" pod "1c74e310-6638-4b15-97d8-65854946086a" (UID: "1c74e310-6638-4b15-97d8-65854946086a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:21:26.420457 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:26.420237    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/bd83c13a-8f22-4ada-a894-90cd49e87782-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bd83c13a-8f22-4ada-a894-90cd49e87782-istiod-ca-cert\") pod \"bd83c13a-8f22-4ada-a894-90cd49e87782\" (UID: \"bd83c13a-8f22-4ada-a894-90cd49e87782\") "
Oct 30 13:21:26.420457 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:26.420280    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/258db5e7-aa22-4bb6-b8ee-844241ca5399-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/258db5e7-aa22-4bb6-b8ee-844241ca5399-istio-podinfo\") pod \"258db5e7-aa22-4bb6-b8ee-844241ca5399\" (UID: \"258db5e7-aa22-4bb6-b8ee-844241ca5399\") "
Oct 30 13:21:26.420457 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:26.420339    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bd83c13a-8f22-4ada-a894-90cd49e87782-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bd83c13a-8f22-4ada-a894-90cd49e87782-istio-data\") pod \"bd83c13a-8f22-4ada-a894-90cd49e87782\" (UID: \"bd83c13a-8f22-4ada-a894-90cd49e87782\") "
Oct 30 13:21:26.420457 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:26.420368    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/bd83c13a-8f22-4ada-a894-90cd49e87782-istio-token\" (UniqueName: \"kubernetes.io/projected/bd83c13a-8f22-4ada-a894-90cd49e87782-istio-token\") pod \"bd83c13a-8f22-4ada-a894-90cd49e87782\" (UID: \"bd83c13a-8f22-4ada-a894-90cd49e87782\") "
Oct 30 13:21:26.420457 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:26.420423    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/258db5e7-aa22-4bb6-b8ee-844241ca5399-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/258db5e7-aa22-4bb6-b8ee-844241ca5399-credential-socket\") pod \"258db5e7-aa22-4bb6-b8ee-844241ca5399\" (UID: \"258db5e7-aa22-4bb6-b8ee-844241ca5399\") "
Oct 30 13:21:26.420771 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:26.420493    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1c74e310-6638-4b15-97d8-65854946086a-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1c74e310-6638-4b15-97d8-65854946086a-workload-certs\") pod \"1c74e310-6638-4b15-97d8-65854946086a\" (UID: \"1c74e310-6638-4b15-97d8-65854946086a\") "
Oct 30 13:21:26.420771 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:26.420530    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bd83c13a-8f22-4ada-a894-90cd49e87782-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bd83c13a-8f22-4ada-a894-90cd49e87782-workload-certs\") pod \"bd83c13a-8f22-4ada-a894-90cd49e87782\" (UID: \"bd83c13a-8f22-4ada-a894-90cd49e87782\") "
Oct 30 13:21:26.420771 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:26.420560    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/258db5e7-aa22-4bb6-b8ee-844241ca5399-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/258db5e7-aa22-4bb6-b8ee-844241ca5399-istio-envoy\") pod \"258db5e7-aa22-4bb6-b8ee-844241ca5399\" (UID: \"258db5e7-aa22-4bb6-b8ee-844241ca5399\") "
Oct 30 13:21:26.420771 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:26.420592    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bd83c13a-8f22-4ada-a894-90cd49e87782-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bd83c13a-8f22-4ada-a894-90cd49e87782-istio-envoy\") pod \"bd83c13a-8f22-4ada-a894-90cd49e87782\" (UID: \"bd83c13a-8f22-4ada-a894-90cd49e87782\") "
Oct 30 13:21:26.420771 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:26.420632    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1c74e310-6638-4b15-97d8-65854946086a-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1c74e310-6638-4b15-97d8-65854946086a-credential-socket\") pod \"1c74e310-6638-4b15-97d8-65854946086a\" (UID: \"1c74e310-6638-4b15-97d8-65854946086a\") "
Oct 30 13:21:26.420771 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:26.420656    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1c74e310-6638-4b15-97d8-65854946086a-istiod-ca-cert" pod "1c74e310-6638-4b15-97d8-65854946086a" (UID: "1c74e310-6638-4b15-97d8-65854946086a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 13:21:26.420771 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:26.420673    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/258db5e7-aa22-4bb6-b8ee-844241ca5399-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/258db5e7-aa22-4bb6-b8ee-844241ca5399-workload-socket\") pod \"258db5e7-aa22-4bb6-b8ee-844241ca5399\" (UID: \"258db5e7-aa22-4bb6-b8ee-844241ca5399\") "
Oct 30 13:21:26.420771 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:26.420760    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/258db5e7-aa22-4bb6-b8ee-844241ca5399-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/258db5e7-aa22-4bb6-b8ee-844241ca5399-istio-data\") pod \"258db5e7-aa22-4bb6-b8ee-844241ca5399\" (UID: \"258db5e7-aa22-4bb6-b8ee-844241ca5399\") "
Oct 30 13:21:26.420771 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:26.420769    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/258db5e7-aa22-4bb6-b8ee-844241ca5399-istiod-ca-cert" pod "258db5e7-aa22-4bb6-b8ee-844241ca5399" (UID: "258db5e7-aa22-4bb6-b8ee-844241ca5399"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 13:21:26.421315 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:26.420865    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/258db5e7-aa22-4bb6-b8ee-844241ca5399-workload-socket" pod "258db5e7-aa22-4bb6-b8ee-844241ca5399" (UID: "258db5e7-aa22-4bb6-b8ee-844241ca5399"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:21:26.421315 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:26.420896    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1c74e310-6638-4b15-97d8-65854946086a-istio-token\" (UniqueName: \"kubernetes.io/projected/1c74e310-6638-4b15-97d8-65854946086a-istio-token\") pod \"1c74e310-6638-4b15-97d8-65854946086a\" (UID: \"1c74e310-6638-4b15-97d8-65854946086a\") "
Oct 30 13:21:26.421315 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:26.421043    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1c74e310-6638-4b15-97d8-65854946086a-istio-data" pod "1c74e310-6638-4b15-97d8-65854946086a" (UID: "1c74e310-6638-4b15-97d8-65854946086a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:21:26.421315 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:26.421242    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/258db5e7-aa22-4bb6-b8ee-844241ca5399-credential-socket" pod "258db5e7-aa22-4bb6-b8ee-844241ca5399" (UID: "258db5e7-aa22-4bb6-b8ee-844241ca5399"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:21:26.421527 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:26.421434    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1c74e310-6638-4b15-97d8-65854946086a-workload-certs" pod "1c74e310-6638-4b15-97d8-65854946086a" (UID: "1c74e310-6638-4b15-97d8-65854946086a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:21:26.421924 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:26.421686    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bd83c13a-8f22-4ada-a894-90cd49e87782-workload-certs" pod "bd83c13a-8f22-4ada-a894-90cd49e87782" (UID: "bd83c13a-8f22-4ada-a894-90cd49e87782"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:21:26.424926 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:26.422422    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/258db5e7-aa22-4bb6-b8ee-844241ca5399-istio-data" pod "258db5e7-aa22-4bb6-b8ee-844241ca5399" (UID: "258db5e7-aa22-4bb6-b8ee-844241ca5399"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:21:26.424926 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:26.422599    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1c74e310-6638-4b15-97d8-65854946086a-credential-socket" pod "1c74e310-6638-4b15-97d8-65854946086a" (UID: "1c74e310-6638-4b15-97d8-65854946086a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:21:26.424926 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:26.423127    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bd83c13a-8f22-4ada-a894-90cd49e87782-credential-socket" pod "bd83c13a-8f22-4ada-a894-90cd49e87782" (UID: "bd83c13a-8f22-4ada-a894-90cd49e87782"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:21:26.424926 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:26.423311    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bd83c13a-8f22-4ada-a894-90cd49e87782-istio-data" pod "bd83c13a-8f22-4ada-a894-90cd49e87782" (UID: "bd83c13a-8f22-4ada-a894-90cd49e87782"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:21:26.424926 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:26.423473    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/258db5e7-aa22-4bb6-b8ee-844241ca5399-kube-api-access-vxwjt\" (UniqueName: \"kubernetes.io/projected/258db5e7-aa22-4bb6-b8ee-844241ca5399-kube-api-access-vxwjt\") pod \"258db5e7-aa22-4bb6-b8ee-844241ca5399\" (UID: \"258db5e7-aa22-4bb6-b8ee-844241ca5399\") "
Oct 30 13:21:26.424926 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:26.423900    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bd83c13a-8f22-4ada-a894-90cd49e87782-istiod-ca-cert" pod "bd83c13a-8f22-4ada-a894-90cd49e87782" (UID: "bd83c13a-8f22-4ada-a894-90cd49e87782"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 13:21:26.424926 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:26.424086    4943 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/258db5e7-aa22-4bb6-b8ee-844241ca5399-credential-socket\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:21:26.424926 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:26.424108    4943 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1c74e310-6638-4b15-97d8-65854946086a-workload-certs\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:21:26.424926 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:26.424123    4943 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bd83c13a-8f22-4ada-a894-90cd49e87782-workload-certs\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:21:26.424926 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:26.424137    4943 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1c74e310-6638-4b15-97d8-65854946086a-credential-socket\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:21:26.424926 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:26.424152    4943 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/258db5e7-aa22-4bb6-b8ee-844241ca5399-workload-socket\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:21:26.424926 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:26.424165    4943 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/258db5e7-aa22-4bb6-b8ee-844241ca5399-istio-data\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:21:26.424926 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:26.424179    4943 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/258db5e7-aa22-4bb6-b8ee-844241ca5399-istiod-ca-cert\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:21:26.424926 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:26.424191    4943 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/258db5e7-aa22-4bb6-b8ee-844241ca5399-workload-certs\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:21:26.424926 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:26.424204    4943 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1c74e310-6638-4b15-97d8-65854946086a-workload-socket\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:21:26.424926 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:26.424218    4943 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1c74e310-6638-4b15-97d8-65854946086a-istiod-ca-cert\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:21:26.424926 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:26.424233    4943 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1c74e310-6638-4b15-97d8-65854946086a-istio-data\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:21:26.424926 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:26.424246    4943 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bd83c13a-8f22-4ada-a894-90cd49e87782-credential-socket\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:21:26.424926 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:26.424276    4943 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bd83c13a-8f22-4ada-a894-90cd49e87782-istiod-ca-cert\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:21:26.424926 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:26.424290    4943 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bd83c13a-8f22-4ada-a894-90cd49e87782-istio-data\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:21:26.424926 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:26.424342    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bd83c13a-8f22-4ada-a894-90cd49e87782-workload-socket" pod "bd83c13a-8f22-4ada-a894-90cd49e87782" (UID: "bd83c13a-8f22-4ada-a894-90cd49e87782"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:21:26.429275 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:26.427166    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/258db5e7-aa22-4bb6-b8ee-844241ca5399-istio-podinfo" pod "258db5e7-aa22-4bb6-b8ee-844241ca5399" (UID: "258db5e7-aa22-4bb6-b8ee-844241ca5399"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 13:21:26.429275 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:26.427865    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bd83c13a-8f22-4ada-a894-90cd49e87782-istio-envoy" pod "bd83c13a-8f22-4ada-a894-90cd49e87782" (UID: "bd83c13a-8f22-4ada-a894-90cd49e87782"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:21:26.429275 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:26.428058    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bd83c13a-8f22-4ada-a894-90cd49e87782-istio-token" pod "bd83c13a-8f22-4ada-a894-90cd49e87782" (UID: "bd83c13a-8f22-4ada-a894-90cd49e87782"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:21:26.431617 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:26.429495    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bd83c13a-8f22-4ada-a894-90cd49e87782-kube-api-access-xlprp" pod "bd83c13a-8f22-4ada-a894-90cd49e87782" (UID: "bd83c13a-8f22-4ada-a894-90cd49e87782"). InnerVolumeSpecName "kube-api-access-xlprp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:21:26.431617 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:26.429608    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/258db5e7-aa22-4bb6-b8ee-844241ca5399-istio-token" pod "258db5e7-aa22-4bb6-b8ee-844241ca5399" (UID: "258db5e7-aa22-4bb6-b8ee-844241ca5399"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:21:26.431617 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:26.430713    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1c74e310-6638-4b15-97d8-65854946086a-kube-api-access-5clq5" pod "1c74e310-6638-4b15-97d8-65854946086a" (UID: "1c74e310-6638-4b15-97d8-65854946086a"). InnerVolumeSpecName "kube-api-access-5clq5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:21:26.431617 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:26.431043    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/258db5e7-aa22-4bb6-b8ee-844241ca5399-istio-envoy" pod "258db5e7-aa22-4bb6-b8ee-844241ca5399" (UID: "258db5e7-aa22-4bb6-b8ee-844241ca5399"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:21:26.431617 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:26.431119    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/bd83c13a-8f22-4ada-a894-90cd49e87782-istio-podinfo" pod "bd83c13a-8f22-4ada-a894-90cd49e87782" (UID: "bd83c13a-8f22-4ada-a894-90cd49e87782"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 13:21:26.432652 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:26.432627    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1c74e310-6638-4b15-97d8-65854946086a-istio-token" pod "1c74e310-6638-4b15-97d8-65854946086a" (UID: "1c74e310-6638-4b15-97d8-65854946086a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:21:26.432767 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:26.432734    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/1c74e310-6638-4b15-97d8-65854946086a-istio-podinfo" pod "1c74e310-6638-4b15-97d8-65854946086a" (UID: "1c74e310-6638-4b15-97d8-65854946086a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 13:21:26.433447 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:26.433418    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1c74e310-6638-4b15-97d8-65854946086a-istio-envoy" pod "1c74e310-6638-4b15-97d8-65854946086a" (UID: "1c74e310-6638-4b15-97d8-65854946086a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:21:26.433662 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:26.433644    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/258db5e7-aa22-4bb6-b8ee-844241ca5399-kube-api-access-vxwjt" pod "258db5e7-aa22-4bb6-b8ee-844241ca5399" (UID: "258db5e7-aa22-4bb6-b8ee-844241ca5399"). InnerVolumeSpecName "kube-api-access-vxwjt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:21:26.449631 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:26.449605    4943 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w4-88f6a4-7dd558b944-gkzck"
Oct 30 13:21:26.465798 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:26.465771    4943 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-ljkqn"
Oct 30 13:21:26.467754 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:26.467733    4943 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-kpvbd"
Oct 30 13:21:26.524587 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:26.524560    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/80c84ecd-0a11-4d8e-ab46-2ba9efd4e676-kube-api-access-l7t5v\" (UniqueName: \"kubernetes.io/projected/80c84ecd-0a11-4d8e-ab46-2ba9efd4e676-kube-api-access-l7t5v\") pod \"80c84ecd-0a11-4d8e-ab46-2ba9efd4e676\" (UID: \"80c84ecd-0a11-4d8e-ab46-2ba9efd4e676\") "
Oct 30 13:21:26.524722 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:26.524593    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f0e82596-0383-4ec9-a856-a6234e9da808-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f0e82596-0383-4ec9-a856-a6234e9da808-credential-socket\") pod \"f0e82596-0383-4ec9-a856-a6234e9da808\" (UID: \"f0e82596-0383-4ec9-a856-a6234e9da808\") "
Oct 30 13:21:26.524722 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:26.524641    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f0ab36b0-4ba8-41da-8fc2-6a70d1082599-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f0ab36b0-4ba8-41da-8fc2-6a70d1082599-workload-socket\") pod \"f0ab36b0-4ba8-41da-8fc2-6a70d1082599\" (UID: \"f0ab36b0-4ba8-41da-8fc2-6a70d1082599\") "
Oct 30 13:21:26.526371 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:26.524726    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f0ab36b0-4ba8-41da-8fc2-6a70d1082599-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f0ab36b0-4ba8-41da-8fc2-6a70d1082599-workload-certs\") pod \"f0ab36b0-4ba8-41da-8fc2-6a70d1082599\" (UID: \"f0ab36b0-4ba8-41da-8fc2-6a70d1082599\") "
Oct 30 13:21:26.526371 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:26.524776    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/80c84ecd-0a11-4d8e-ab46-2ba9efd4e676-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/80c84ecd-0a11-4d8e-ab46-2ba9efd4e676-plugins-config-volume\") pod \"80c84ecd-0a11-4d8e-ab46-2ba9efd4e676\" (UID: \"80c84ecd-0a11-4d8e-ab46-2ba9efd4e676\") "
Oct 30 13:21:26.526371 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:26.524798    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f0e82596-0383-4ec9-a856-a6234e9da808-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f0e82596-0383-4ec9-a856-a6234e9da808-workload-socket\") pod \"f0e82596-0383-4ec9-a856-a6234e9da808\" (UID: \"f0e82596-0383-4ec9-a856-a6234e9da808\") "
Oct 30 13:21:26.526371 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:26.524817    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f0ab36b0-4ba8-41da-8fc2-6a70d1082599-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f0ab36b0-4ba8-41da-8fc2-6a70d1082599-istio-envoy\") pod \"f0ab36b0-4ba8-41da-8fc2-6a70d1082599\" (UID: \"f0ab36b0-4ba8-41da-8fc2-6a70d1082599\") "
Oct 30 13:21:26.526371 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:26.524869    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f0ab36b0-4ba8-41da-8fc2-6a70d1082599-workload-socket" pod "f0ab36b0-4ba8-41da-8fc2-6a70d1082599" (UID: "f0ab36b0-4ba8-41da-8fc2-6a70d1082599"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:21:26.526371 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:26.524866    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f0e82596-0383-4ec9-a856-a6234e9da808-istio-token\" (UniqueName: \"kubernetes.io/projected/f0e82596-0383-4ec9-a856-a6234e9da808-istio-token\") pod \"f0e82596-0383-4ec9-a856-a6234e9da808\" (UID: \"f0e82596-0383-4ec9-a856-a6234e9da808\") "
Oct 30 13:21:26.526371 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:26.524921    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f0e82596-0383-4ec9-a856-a6234e9da808-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f0e82596-0383-4ec9-a856-a6234e9da808-workload-certs\") pod \"f0e82596-0383-4ec9-a856-a6234e9da808\" (UID: \"f0e82596-0383-4ec9-a856-a6234e9da808\") "
Oct 30 13:21:26.526371 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:26.524920    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f0e82596-0383-4ec9-a856-a6234e9da808-credential-socket" pod "f0e82596-0383-4ec9-a856-a6234e9da808" (UID: "f0e82596-0383-4ec9-a856-a6234e9da808"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:21:26.526371 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:26.525014    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f0e82596-0383-4ec9-a856-a6234e9da808-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f0e82596-0383-4ec9-a856-a6234e9da808-istio-envoy\") pod \"f0e82596-0383-4ec9-a856-a6234e9da808\" (UID: \"f0e82596-0383-4ec9-a856-a6234e9da808\") "
Oct 30 13:21:26.526371 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:26.525030    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f0ab36b0-4ba8-41da-8fc2-6a70d1082599-workload-certs" pod "f0ab36b0-4ba8-41da-8fc2-6a70d1082599" (UID: "f0ab36b0-4ba8-41da-8fc2-6a70d1082599"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:21:26.526371 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:26.525058    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f0e82596-0383-4ec9-a856-a6234e9da808-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f0e82596-0383-4ec9-a856-a6234e9da808-istio-data\") pod \"f0e82596-0383-4ec9-a856-a6234e9da808\" (UID: \"f0e82596-0383-4ec9-a856-a6234e9da808\") "
Oct 30 13:21:26.526371 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:26.525098    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f0e82596-0383-4ec9-a856-a6234e9da808-workload-socket" pod "f0e82596-0383-4ec9-a856-a6234e9da808" (UID: "f0e82596-0383-4ec9-a856-a6234e9da808"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:21:26.526371 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:26.525101    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/f0e82596-0383-4ec9-a856-a6234e9da808-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f0e82596-0383-4ec9-a856-a6234e9da808-istio-podinfo\") pod \"f0e82596-0383-4ec9-a856-a6234e9da808\" (UID: \"f0e82596-0383-4ec9-a856-a6234e9da808\") "
Oct 30 13:21:26.526371 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:26.525148    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f0ab36b0-4ba8-41da-8fc2-6a70d1082599-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f0ab36b0-4ba8-41da-8fc2-6a70d1082599-istio-data\") pod \"f0ab36b0-4ba8-41da-8fc2-6a70d1082599\" (UID: \"f0ab36b0-4ba8-41da-8fc2-6a70d1082599\") "
Oct 30 13:21:26.526371 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:26.525170    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f0e82596-0383-4ec9-a856-a6234e9da808-workload-certs" pod "f0e82596-0383-4ec9-a856-a6234e9da808" (UID: "f0e82596-0383-4ec9-a856-a6234e9da808"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:21:26.526371 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:26.525187    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f0ab36b0-4ba8-41da-8fc2-6a70d1082599-istio-token\" (UniqueName: \"kubernetes.io/projected/f0ab36b0-4ba8-41da-8fc2-6a70d1082599-istio-token\") pod \"f0ab36b0-4ba8-41da-8fc2-6a70d1082599\" (UID: \"f0ab36b0-4ba8-41da-8fc2-6a70d1082599\") "
Oct 30 13:21:26.526371 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:26.525231    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f0e82596-0383-4ec9-a856-a6234e9da808-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f0e82596-0383-4ec9-a856-a6234e9da808-istiod-ca-cert\") pod \"f0e82596-0383-4ec9-a856-a6234e9da808\" (UID: \"f0e82596-0383-4ec9-a856-a6234e9da808\") "
Oct 30 13:21:26.526371 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:26.525243    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/80c84ecd-0a11-4d8e-ab46-2ba9efd4e676-plugins-config-volume" pod "80c84ecd-0a11-4d8e-ab46-2ba9efd4e676" (UID: "80c84ecd-0a11-4d8e-ab46-2ba9efd4e676"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 13:21:26.526371 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:26.525305    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f0ab36b0-4ba8-41da-8fc2-6a70d1082599-kube-api-access-k2cbg\" (UniqueName: \"kubernetes.io/projected/f0ab36b0-4ba8-41da-8fc2-6a70d1082599-kube-api-access-k2cbg\") pod \"f0ab36b0-4ba8-41da-8fc2-6a70d1082599\" (UID: \"f0ab36b0-4ba8-41da-8fc2-6a70d1082599\") "
Oct 30 13:21:26.526371 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:26.525405    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f0ab36b0-4ba8-41da-8fc2-6a70d1082599-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f0ab36b0-4ba8-41da-8fc2-6a70d1082599-istiod-ca-cert\") pod \"f0ab36b0-4ba8-41da-8fc2-6a70d1082599\" (UID: \"f0ab36b0-4ba8-41da-8fc2-6a70d1082599\") "
Oct 30 13:21:26.526371 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:26.525436    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f0ab36b0-4ba8-41da-8fc2-6a70d1082599-istio-data" pod "f0ab36b0-4ba8-41da-8fc2-6a70d1082599" (UID: "f0ab36b0-4ba8-41da-8fc2-6a70d1082599"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:21:26.526371 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:26.525449    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f0e82596-0383-4ec9-a856-a6234e9da808-kube-api-access-mhwd5\" (UniqueName: \"kubernetes.io/projected/f0e82596-0383-4ec9-a856-a6234e9da808-kube-api-access-mhwd5\") pod \"f0e82596-0383-4ec9-a856-a6234e9da808\" (UID: \"f0e82596-0383-4ec9-a856-a6234e9da808\") "
Oct 30 13:21:26.526371 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:26.525524    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/f0ab36b0-4ba8-41da-8fc2-6a70d1082599-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f0ab36b0-4ba8-41da-8fc2-6a70d1082599-istio-podinfo\") pod \"f0ab36b0-4ba8-41da-8fc2-6a70d1082599\" (UID: \"f0ab36b0-4ba8-41da-8fc2-6a70d1082599\") "
Oct 30 13:21:26.526371 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:26.525553    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f0ab36b0-4ba8-41da-8fc2-6a70d1082599-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f0ab36b0-4ba8-41da-8fc2-6a70d1082599-credential-socket\") pod \"f0ab36b0-4ba8-41da-8fc2-6a70d1082599\" (UID: \"f0ab36b0-4ba8-41da-8fc2-6a70d1082599\") "
Oct 30 13:21:26.526371 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:26.525661    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f0e82596-0383-4ec9-a856-a6234e9da808-istio-data" pod "f0e82596-0383-4ec9-a856-a6234e9da808" (UID: "f0e82596-0383-4ec9-a856-a6234e9da808"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:21:26.526371 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:26.525920    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f0ab36b0-4ba8-41da-8fc2-6a70d1082599-credential-socket" pod "f0ab36b0-4ba8-41da-8fc2-6a70d1082599" (UID: "f0ab36b0-4ba8-41da-8fc2-6a70d1082599"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:21:26.526371 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:26.525991    4943 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1c74e310-6638-4b15-97d8-65854946086a-istio-token\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:21:26.526371 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:26.526015    4943 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vxwjt\" (UniqueName: \"kubernetes.io/projected/258db5e7-aa22-4bb6-b8ee-844241ca5399-kube-api-access-vxwjt\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:21:26.526371 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:26.526031    4943 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5clq5\" (UniqueName: \"kubernetes.io/projected/1c74e310-6638-4b15-97d8-65854946086a-kube-api-access-5clq5\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:21:26.526371 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:26.526046    4943 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f0e82596-0383-4ec9-a856-a6234e9da808-credential-socket\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:21:26.526371 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:26.526061    4943 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1c74e310-6638-4b15-97d8-65854946086a-istio-podinfo\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:21:26.526371 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:26.526075    4943 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bd83c13a-8f22-4ada-a894-90cd49e87782-workload-socket\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:21:26.526371 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:26.526089    4943 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f0ab36b0-4ba8-41da-8fc2-6a70d1082599-workload-socket\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:21:26.526371 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:26.526104    4943 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/258db5e7-aa22-4bb6-b8ee-844241ca5399-istio-token\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:21:26.526371 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:26.526116    4943 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f0ab36b0-4ba8-41da-8fc2-6a70d1082599-workload-certs\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:21:26.526371 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:26.526130    4943 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xlprp\" (UniqueName: \"kubernetes.io/projected/bd83c13a-8f22-4ada-a894-90cd49e87782-kube-api-access-xlprp\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:21:26.526371 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:26.526146    4943 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/80c84ecd-0a11-4d8e-ab46-2ba9efd4e676-plugins-config-volume\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:21:26.526371 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:26.526161    4943 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1c74e310-6638-4b15-97d8-65854946086a-istio-envoy\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:21:26.526371 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:26.526175    4943 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f0e82596-0383-4ec9-a856-a6234e9da808-workload-socket\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:21:26.526371 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:26.526188    4943 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f0e82596-0383-4ec9-a856-a6234e9da808-workload-certs\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:21:26.526371 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:26.526202    4943 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bd83c13a-8f22-4ada-a894-90cd49e87782-istio-podinfo\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:21:26.526371 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:26.526215    4943 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/258db5e7-aa22-4bb6-b8ee-844241ca5399-istio-podinfo\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:21:26.526371 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:26.526217    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f0e82596-0383-4ec9-a856-a6234e9da808-istiod-ca-cert" pod "f0e82596-0383-4ec9-a856-a6234e9da808" (UID: "f0e82596-0383-4ec9-a856-a6234e9da808"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 13:21:26.526371 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:26.526227    4943 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f0e82596-0383-4ec9-a856-a6234e9da808-istio-data\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:21:26.526371 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:26.526275    4943 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bd83c13a-8f22-4ada-a894-90cd49e87782-istio-token\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:21:26.526371 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:26.526291    4943 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f0ab36b0-4ba8-41da-8fc2-6a70d1082599-istio-data\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:21:26.526371 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:26.526304    4943 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/258db5e7-aa22-4bb6-b8ee-844241ca5399-istio-envoy\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:21:26.526371 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:26.526317    4943 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bd83c13a-8f22-4ada-a894-90cd49e87782-istio-envoy\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:21:26.529108 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:26.526462    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f0ab36b0-4ba8-41da-8fc2-6a70d1082599-istiod-ca-cert" pod "f0ab36b0-4ba8-41da-8fc2-6a70d1082599" (UID: "f0ab36b0-4ba8-41da-8fc2-6a70d1082599"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 13:21:26.529108 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:26.527438    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/80c84ecd-0a11-4d8e-ab46-2ba9efd4e676-kube-api-access-l7t5v" pod "80c84ecd-0a11-4d8e-ab46-2ba9efd4e676" (UID: "80c84ecd-0a11-4d8e-ab46-2ba9efd4e676"). InnerVolumeSpecName "kube-api-access-l7t5v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:21:26.529108 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:26.527632    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f0ab36b0-4ba8-41da-8fc2-6a70d1082599-istio-token" pod "f0ab36b0-4ba8-41da-8fc2-6a70d1082599" (UID: "f0ab36b0-4ba8-41da-8fc2-6a70d1082599"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:21:26.529108 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:26.527886    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f0ab36b0-4ba8-41da-8fc2-6a70d1082599-kube-api-access-k2cbg" pod "f0ab36b0-4ba8-41da-8fc2-6a70d1082599" (UID: "f0ab36b0-4ba8-41da-8fc2-6a70d1082599"). InnerVolumeSpecName "kube-api-access-k2cbg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:21:26.529108 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:26.528316    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f0ab36b0-4ba8-41da-8fc2-6a70d1082599-istio-envoy" pod "f0ab36b0-4ba8-41da-8fc2-6a70d1082599" (UID: "f0ab36b0-4ba8-41da-8fc2-6a70d1082599"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:21:26.529108 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:26.528418    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f0e82596-0383-4ec9-a856-a6234e9da808-istio-token" pod "f0e82596-0383-4ec9-a856-a6234e9da808" (UID: "f0e82596-0383-4ec9-a856-a6234e9da808"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:21:26.529108 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:26.528797    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f0e82596-0383-4ec9-a856-a6234e9da808-istio-podinfo" pod "f0e82596-0383-4ec9-a856-a6234e9da808" (UID: "f0e82596-0383-4ec9-a856-a6234e9da808"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 13:21:26.529404 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:26.529240    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f0e82596-0383-4ec9-a856-a6234e9da808-istio-envoy" pod "f0e82596-0383-4ec9-a856-a6234e9da808" (UID: "f0e82596-0383-4ec9-a856-a6234e9da808"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:21:26.529404 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:26.529396    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f0ab36b0-4ba8-41da-8fc2-6a70d1082599-istio-podinfo" pod "f0ab36b0-4ba8-41da-8fc2-6a70d1082599" (UID: "f0ab36b0-4ba8-41da-8fc2-6a70d1082599"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 13:21:26.529744 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:26.529729    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f0e82596-0383-4ec9-a856-a6234e9da808-kube-api-access-mhwd5" pod "f0e82596-0383-4ec9-a856-a6234e9da808" (UID: "f0e82596-0383-4ec9-a856-a6234e9da808"). InnerVolumeSpecName "kube-api-access-mhwd5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:21:26.626976 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:26.626942    4943 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f0ab36b0-4ba8-41da-8fc2-6a70d1082599-istio-podinfo\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:21:26.626976 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:26.626971    4943 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f0ab36b0-4ba8-41da-8fc2-6a70d1082599-credential-socket\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:21:26.626976 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:26.626981    4943 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l7t5v\" (UniqueName: \"kubernetes.io/projected/80c84ecd-0a11-4d8e-ab46-2ba9efd4e676-kube-api-access-l7t5v\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:21:26.627419 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:26.626991    4943 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f0ab36b0-4ba8-41da-8fc2-6a70d1082599-istio-envoy\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:21:26.627419 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:26.627000    4943 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f0e82596-0383-4ec9-a856-a6234e9da808-istio-token\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:21:26.627419 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:26.627008    4943 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f0e82596-0383-4ec9-a856-a6234e9da808-istio-envoy\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:21:26.627419 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:26.627016    4943 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f0e82596-0383-4ec9-a856-a6234e9da808-istio-podinfo\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:21:26.627419 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:26.627024    4943 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f0ab36b0-4ba8-41da-8fc2-6a70d1082599-istio-token\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:21:26.627419 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:26.627033    4943 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f0e82596-0383-4ec9-a856-a6234e9da808-istiod-ca-cert\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:21:26.627419 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:26.627042    4943 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k2cbg\" (UniqueName: \"kubernetes.io/projected/f0ab36b0-4ba8-41da-8fc2-6a70d1082599-kube-api-access-k2cbg\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:21:26.627419 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:26.627050    4943 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f0ab36b0-4ba8-41da-8fc2-6a70d1082599-istiod-ca-cert\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:21:26.627419 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:26.627059    4943 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mhwd5\" (UniqueName: \"kubernetes.io/projected/f0e82596-0383-4ec9-a856-a6234e9da808-kube-api-access-mhwd5\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:21:26.919604 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:26.919563    4943 generic.go:356] "Generic (PLEG): container finished" podID="258db5e7-aa22-4bb6-b8ee-844241ca5399" containerID="9989ec3323990145f3437b91850185cbffab2c42b9e6f132674e969ffb52b858" exitCode=0
Oct 30 13:21:26.919783 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:26.919606    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-bf7pz" event={"ID":"258db5e7-aa22-4bb6-b8ee-844241ca5399","Type":"ContainerDied","Data":"9989ec3323990145f3437b91850185cbffab2c42b9e6f132674e969ffb52b858"}
Oct 30 13:21:26.919783 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:26.919619    4943 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-bf7pz"
Oct 30 13:21:26.919783 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:26.919676    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-bf7pz" event={"ID":"258db5e7-aa22-4bb6-b8ee-844241ca5399","Type":"ContainerDied","Data":"293c778564cc8d05bc32dc0c70ae0ce60b0f8c4357a415370181aefd18320b28"}
Oct 30 13:21:26.919783 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:26.919708    4943 scope.go:122] "RemoveContainer" containerID="9989ec3323990145f3437b91850185cbffab2c42b9e6f132674e969ffb52b858"
Oct 30 13:21:26.921546 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:26.921523    4943 generic.go:356] "Generic (PLEG): container finished" podID="1c74e310-6638-4b15-97d8-65854946086a" containerID="343c2ddb8a87c1a0d9493df064e5f7fb86fe37d31cd4e3664dfeadb6b3cc133b" exitCode=0
Oct 30 13:21:26.921660 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:26.921570    4943 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-mlh2m"
Oct 30 13:21:26.921660 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:26.921611    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-mlh2m" event={"ID":"1c74e310-6638-4b15-97d8-65854946086a","Type":"ContainerDied","Data":"343c2ddb8a87c1a0d9493df064e5f7fb86fe37d31cd4e3664dfeadb6b3cc133b"}
Oct 30 13:21:26.921778 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:26.921666    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-mlh2m" event={"ID":"1c74e310-6638-4b15-97d8-65854946086a","Type":"ContainerDied","Data":"d1dd8a63c5e7a1a8fb3d8ed87cc006c0f693bcc3da2ca5b56f1d4ea534058f91"}
Oct 30 13:21:26.924991 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:26.924964    4943 generic.go:356] "Generic (PLEG): container finished" podID="f0e82596-0383-4ec9-a856-a6234e9da808" containerID="0717bcb2ccdab96dd7a6937fd08a0fe5581f31152a2d124a5306480719d7c3f7" exitCode=0
Oct 30 13:21:26.925119 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:26.925019    4943 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-kpvbd"
Oct 30 13:21:26.925119 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:26.925047    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-kpvbd" event={"ID":"f0e82596-0383-4ec9-a856-a6234e9da808","Type":"ContainerDied","Data":"0717bcb2ccdab96dd7a6937fd08a0fe5581f31152a2d124a5306480719d7c3f7"}
Oct 30 13:21:26.925119 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:26.925090    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-kpvbd" event={"ID":"f0e82596-0383-4ec9-a856-a6234e9da808","Type":"ContainerDied","Data":"ed430c976530b0ea483a00e6f204a2c40d441a40cab4b6132652269f854fc246"}
Oct 30 13:21:26.927353 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:26.927331    4943 generic.go:356] "Generic (PLEG): container finished" podID="f0ab36b0-4ba8-41da-8fc2-6a70d1082599" containerID="3bb47869cf566fc54232cdffa43db4a7042cefe5994382c6cec1269971649d61" exitCode=0
Oct 30 13:21:26.927485 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:26.927372    4943 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-ljkqn"
Oct 30 13:21:26.927485 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:26.927413    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-ljkqn" event={"ID":"f0ab36b0-4ba8-41da-8fc2-6a70d1082599","Type":"ContainerDied","Data":"3bb47869cf566fc54232cdffa43db4a7042cefe5994382c6cec1269971649d61"}
Oct 30 13:21:26.927485 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:26.927473    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-ljkqn" event={"ID":"f0ab36b0-4ba8-41da-8fc2-6a70d1082599","Type":"ContainerDied","Data":"1a0d458f0843f21a6f358dd72f69723eda6aad9fc1cdd4c569327c01bc4921be"}
Oct 30 13:21:26.929989 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:26.929964    4943 generic.go:356] "Generic (PLEG): container finished" podID="bd83c13a-8f22-4ada-a894-90cd49e87782" containerID="c3864650b233dda9248352e7bb7d5bccbe2291f3806423c81a128038986a9903" exitCode=0
Oct 30 13:21:26.930114 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:26.930066    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-pph7x" event={"ID":"bd83c13a-8f22-4ada-a894-90cd49e87782","Type":"ContainerDied","Data":"c3864650b233dda9248352e7bb7d5bccbe2291f3806423c81a128038986a9903"}
Oct 30 13:21:26.930114 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:26.930106    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-pph7x" event={"ID":"bd83c13a-8f22-4ada-a894-90cd49e87782","Type":"ContainerDied","Data":"5add7b0434636953d0383351f4a97b926da5985db410201394c9d68df1ba57fc"}
Oct 30 13:21:26.930234 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:26.930180    4943 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-pph7x"
Oct 30 13:21:26.936429 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:26.936363    4943 scope.go:122] "RemoveContainer" containerID="9989ec3323990145f3437b91850185cbffab2c42b9e6f132674e969ffb52b858"
Oct 30 13:21:26.937064 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:26.937039    4943 generic.go:356] "Generic (PLEG): container finished" podID="80c84ecd-0a11-4d8e-ab46-2ba9efd4e676" containerID="bd85b10273b50dd2fca89ae5428adcff4261b0824c739153bd1148b75a648e9f" exitCode=0
Oct 30 13:21:26.937206 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:26.937173    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w4-88f6a4-7dd558b944-gkzck" event={"ID":"80c84ecd-0a11-4d8e-ab46-2ba9efd4e676","Type":"ContainerDied","Data":"bd85b10273b50dd2fca89ae5428adcff4261b0824c739153bd1148b75a648e9f"}
Oct 30 13:21:26.937320 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:26.937207    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w4-88f6a4-7dd558b944-gkzck" event={"ID":"80c84ecd-0a11-4d8e-ab46-2ba9efd4e676","Type":"ContainerDied","Data":"26347a503a412a0e520ba88ee140cc39344c1f9049422bd5439af101eeabcbe3"}
Oct 30 13:21:26.937320 ip-10-0-142-84 kubenswrapper[4943]: E1030 13:21:26.937252    4943 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9989ec3323990145f3437b91850185cbffab2c42b9e6f132674e969ffb52b858\": container with ID starting with 9989ec3323990145f3437b91850185cbffab2c42b9e6f132674e969ffb52b858 not found: ID does not exist" containerID="9989ec3323990145f3437b91850185cbffab2c42b9e6f132674e969ffb52b858"
Oct 30 13:21:26.937474 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:26.937313    4943 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9989ec3323990145f3437b91850185cbffab2c42b9e6f132674e969ffb52b858"} err="failed to get container status \"9989ec3323990145f3437b91850185cbffab2c42b9e6f132674e969ffb52b858\": rpc error: code = NotFound desc = could not find container \"9989ec3323990145f3437b91850185cbffab2c42b9e6f132674e969ffb52b858\": container with ID starting with 9989ec3323990145f3437b91850185cbffab2c42b9e6f132674e969ffb52b858 not found: ID does not exist"
Oct 30 13:21:26.937474 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:26.937335    4943 scope.go:122] "RemoveContainer" containerID="343c2ddb8a87c1a0d9493df064e5f7fb86fe37d31cd4e3664dfeadb6b3cc133b"
Oct 30 13:21:26.937474 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:26.937423    4943 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w4-88f6a4-7dd558b944-gkzck"
Oct 30 13:21:26.952779 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:26.952756    4943 scope.go:122] "RemoveContainer" containerID="343c2ddb8a87c1a0d9493df064e5f7fb86fe37d31cd4e3664dfeadb6b3cc133b"
Oct 30 13:21:26.957038 ip-10-0-142-84 kubenswrapper[4943]: E1030 13:21:26.957010    4943 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"343c2ddb8a87c1a0d9493df064e5f7fb86fe37d31cd4e3664dfeadb6b3cc133b\": container with ID starting with 343c2ddb8a87c1a0d9493df064e5f7fb86fe37d31cd4e3664dfeadb6b3cc133b not found: ID does not exist" containerID="343c2ddb8a87c1a0d9493df064e5f7fb86fe37d31cd4e3664dfeadb6b3cc133b"
Oct 30 13:21:26.957156 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:26.957047    4943 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"343c2ddb8a87c1a0d9493df064e5f7fb86fe37d31cd4e3664dfeadb6b3cc133b"} err="failed to get container status \"343c2ddb8a87c1a0d9493df064e5f7fb86fe37d31cd4e3664dfeadb6b3cc133b\": rpc error: code = NotFound desc = could not find container \"343c2ddb8a87c1a0d9493df064e5f7fb86fe37d31cd4e3664dfeadb6b3cc133b\": container with ID starting with 343c2ddb8a87c1a0d9493df064e5f7fb86fe37d31cd4e3664dfeadb6b3cc133b not found: ID does not exist"
Oct 30 13:21:26.957156 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:26.957071    4943 scope.go:122] "RemoveContainer" containerID="0717bcb2ccdab96dd7a6937fd08a0fe5581f31152a2d124a5306480719d7c3f7"
Oct 30 13:21:26.967936 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:26.967797    4943 scope.go:122] "RemoveContainer" containerID="0717bcb2ccdab96dd7a6937fd08a0fe5581f31152a2d124a5306480719d7c3f7"
Oct 30 13:21:26.968128 ip-10-0-142-84 kubenswrapper[4943]: E1030 13:21:26.968108    4943 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0717bcb2ccdab96dd7a6937fd08a0fe5581f31152a2d124a5306480719d7c3f7\": container with ID starting with 0717bcb2ccdab96dd7a6937fd08a0fe5581f31152a2d124a5306480719d7c3f7 not found: ID does not exist" containerID="0717bcb2ccdab96dd7a6937fd08a0fe5581f31152a2d124a5306480719d7c3f7"
Oct 30 13:21:26.968167 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:26.968137    4943 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0717bcb2ccdab96dd7a6937fd08a0fe5581f31152a2d124a5306480719d7c3f7"} err="failed to get container status \"0717bcb2ccdab96dd7a6937fd08a0fe5581f31152a2d124a5306480719d7c3f7\": rpc error: code = NotFound desc = could not find container \"0717bcb2ccdab96dd7a6937fd08a0fe5581f31152a2d124a5306480719d7c3f7\": container with ID starting with 0717bcb2ccdab96dd7a6937fd08a0fe5581f31152a2d124a5306480719d7c3f7 not found: ID does not exist"
Oct 30 13:21:26.968167 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:26.968154    4943 scope.go:122] "RemoveContainer" containerID="3bb47869cf566fc54232cdffa43db4a7042cefe5994382c6cec1269971649d61"
Oct 30 13:21:26.971767 ip-10-0-142-84 kubenswrapper[4943]: E1030 13:21:26.971737    4943 cadvisor_stats_provider.go:569] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1c74e310_6638_4b15_97d8_65854946086a.slice\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1c74e310_6638_4b15_97d8_65854946086a.slice/crio-d1dd8a63c5e7a1a8fb3d8ed87cc006c0f693bcc3da2ca5b56f1d4ea534058f91\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod258db5e7_aa22_4bb6_b8ee_844241ca5399.slice\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod258db5e7_aa22_4bb6_b8ee_844241ca5399.slice/crio-293c778564cc8d05bc32dc0c70ae0ce60b0f8c4357a415370181aefd18320b28\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf0ab36b0_4ba8_41da_8fc2_6a70d1082599.slice\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod80c84ecd_0a11_4d8e_ab46_2ba9efd4e676.slice\": RecentStats: unable to find data in memory cache]"
Oct 30 13:21:26.978005 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:26.977985    4943 scope.go:122] "RemoveContainer" containerID="3bb47869cf566fc54232cdffa43db4a7042cefe5994382c6cec1269971649d61"
Oct 30 13:21:26.978296 ip-10-0-142-84 kubenswrapper[4943]: E1030 13:21:26.978249    4943 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3bb47869cf566fc54232cdffa43db4a7042cefe5994382c6cec1269971649d61\": container with ID starting with 3bb47869cf566fc54232cdffa43db4a7042cefe5994382c6cec1269971649d61 not found: ID does not exist" containerID="3bb47869cf566fc54232cdffa43db4a7042cefe5994382c6cec1269971649d61"
Oct 30 13:21:26.978354 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:26.978305    4943 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3bb47869cf566fc54232cdffa43db4a7042cefe5994382c6cec1269971649d61"} err="failed to get container status \"3bb47869cf566fc54232cdffa43db4a7042cefe5994382c6cec1269971649d61\": rpc error: code = NotFound desc = could not find container \"3bb47869cf566fc54232cdffa43db4a7042cefe5994382c6cec1269971649d61\": container with ID starting with 3bb47869cf566fc54232cdffa43db4a7042cefe5994382c6cec1269971649d61 not found: ID does not exist"
Oct 30 13:21:26.978354 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:26.978322    4943 scope.go:122] "RemoveContainer" containerID="c3864650b233dda9248352e7bb7d5bccbe2291f3806423c81a128038986a9903"
Oct 30 13:21:26.986925 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:26.986906    4943 scope.go:122] "RemoveContainer" containerID="c3864650b233dda9248352e7bb7d5bccbe2291f3806423c81a128038986a9903"
Oct 30 13:21:26.987180 ip-10-0-142-84 kubenswrapper[4943]: E1030 13:21:26.987160    4943 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c3864650b233dda9248352e7bb7d5bccbe2291f3806423c81a128038986a9903\": container with ID starting with c3864650b233dda9248352e7bb7d5bccbe2291f3806423c81a128038986a9903 not found: ID does not exist" containerID="c3864650b233dda9248352e7bb7d5bccbe2291f3806423c81a128038986a9903"
Oct 30 13:21:26.987237 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:26.987185    4943 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c3864650b233dda9248352e7bb7d5bccbe2291f3806423c81a128038986a9903"} err="failed to get container status \"c3864650b233dda9248352e7bb7d5bccbe2291f3806423c81a128038986a9903\": rpc error: code = NotFound desc = could not find container \"c3864650b233dda9248352e7bb7d5bccbe2291f3806423c81a128038986a9903\": container with ID starting with c3864650b233dda9248352e7bb7d5bccbe2291f3806423c81a128038986a9903 not found: ID does not exist"
Oct 30 13:21:26.987237 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:26.987199    4943 scope.go:122] "RemoveContainer" containerID="bd85b10273b50dd2fca89ae5428adcff4261b0824c739153bd1148b75a648e9f"
Oct 30 13:21:26.995845 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:26.995824    4943 scope.go:122] "RemoveContainer" containerID="bd85b10273b50dd2fca89ae5428adcff4261b0824c739153bd1148b75a648e9f"
Oct 30 13:21:26.996121 ip-10-0-142-84 kubenswrapper[4943]: E1030 13:21:26.996095    4943 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bd85b10273b50dd2fca89ae5428adcff4261b0824c739153bd1148b75a648e9f\": container with ID starting with bd85b10273b50dd2fca89ae5428adcff4261b0824c739153bd1148b75a648e9f not found: ID does not exist" containerID="bd85b10273b50dd2fca89ae5428adcff4261b0824c739153bd1148b75a648e9f"
Oct 30 13:21:26.996163 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:26.996130    4943 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bd85b10273b50dd2fca89ae5428adcff4261b0824c739153bd1148b75a648e9f"} err="failed to get container status \"bd85b10273b50dd2fca89ae5428adcff4261b0824c739153bd1148b75a648e9f\": rpc error: code = NotFound desc = could not find container \"bd85b10273b50dd2fca89ae5428adcff4261b0824c739153bd1148b75a648e9f\": container with ID starting with bd85b10273b50dd2fca89ae5428adcff4261b0824c739153bd1148b75a648e9f not found: ID does not exist"
Oct 30 13:21:27.064358 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:27.064329    4943 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-pph7x"]
Oct 30 13:21:27.082158 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:27.082129    4943 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-pph7x"]
Oct 30 13:21:27.129732 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:27.129659    4943 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-kpvbd"]
Oct 30 13:21:27.138988 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:27.138960    4943 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-kpvbd"]
Oct 30 13:21:27.199446 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:27.199421    4943 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-mlh2m"]
Oct 30 13:21:27.213082 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:27.213058    4943 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-mlh2m"]
Oct 30 13:21:27.256438 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:27.256413    4943 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-bf7pz"]
Oct 30 13:21:27.265291 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:27.265251    4943 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-bf7pz"]
Oct 30 13:21:27.304699 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:27.304677    4943 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-ljkqn"]
Oct 30 13:21:27.311763 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:27.311738    4943 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-ljkqn"]
Oct 30 13:21:27.353653 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:27.353631    4943 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-88f6a4-7dd558b944-gkzck"]
Oct 30 13:21:27.362693 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:27.362671    4943 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-88f6a4-7dd558b944-gkzck"]
Oct 30 13:21:28.349910 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:28.349869    4943 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1c74e310-6638-4b15-97d8-65854946086a" path="/var/lib/kubelet/pods/1c74e310-6638-4b15-97d8-65854946086a/volumes"
Oct 30 13:21:28.350533 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:28.350511    4943 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="258db5e7-aa22-4bb6-b8ee-844241ca5399" path="/var/lib/kubelet/pods/258db5e7-aa22-4bb6-b8ee-844241ca5399/volumes"
Oct 30 13:21:28.350933 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:28.350914    4943 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="80c84ecd-0a11-4d8e-ab46-2ba9efd4e676" path="/var/lib/kubelet/pods/80c84ecd-0a11-4d8e-ab46-2ba9efd4e676/volumes"
Oct 30 13:21:28.351640 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:28.351614    4943 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="bd83c13a-8f22-4ada-a894-90cd49e87782" path="/var/lib/kubelet/pods/bd83c13a-8f22-4ada-a894-90cd49e87782/volumes"
Oct 30 13:21:28.352043 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:28.352030    4943 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f0ab36b0-4ba8-41da-8fc2-6a70d1082599" path="/var/lib/kubelet/pods/f0ab36b0-4ba8-41da-8fc2-6a70d1082599/volumes"
Oct 30 13:21:28.352451 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:28.352438    4943 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f0e82596-0383-4ec9-a856-a6234e9da808" path="/var/lib/kubelet/pods/f0e82596-0383-4ec9-a856-a6234e9da808/volumes"
Oct 30 13:21:28.792251 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:28.792204    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-hdmrm"]
Oct 30 13:21:30.199630 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:30.199590    4943 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w6-ab3176-5f84755579-6t85j"]
Oct 30 13:21:30.999545 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:30.999511    4943 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-6631f3dc-vg8sr"]
Oct 30 13:21:30.999961 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:30.999948    4943 cpu_manager.go:414] "containerMap: removing container" podUID="87aca420-1995-4b47-a522-0d60b607db07" containerName="revoke-keys"
Oct 30 13:21:31.000013 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:30.999962    4943 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="87aca420-1995-4b47-a522-0d60b607db07" containerName="revoke-keys"
Oct 30 13:21:31.000013 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:30.999982    4943 cpu_manager.go:414] "containerMap: removing container" podUID="f0ab36b0-4ba8-41da-8fc2-6a70d1082599" containerName="istio-proxy"
Oct 30 13:21:31.000013 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:30.999987    4943 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f0ab36b0-4ba8-41da-8fc2-6a70d1082599" containerName="istio-proxy"
Oct 30 13:21:31.000013 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:31.000006    4943 cpu_manager.go:414] "containerMap: removing container" podUID="1c74e310-6638-4b15-97d8-65854946086a" containerName="istio-proxy"
Oct 30 13:21:31.000013 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:31.000011    4943 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1c74e310-6638-4b15-97d8-65854946086a" containerName="istio-proxy"
Oct 30 13:21:31.000175 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:31.000021    4943 cpu_manager.go:414] "containerMap: removing container" podUID="80c84ecd-0a11-4d8e-ab46-2ba9efd4e676" containerName="payload-processing"
Oct 30 13:21:31.000175 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:31.000026    4943 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="80c84ecd-0a11-4d8e-ab46-2ba9efd4e676" containerName="payload-processing"
Oct 30 13:21:31.000175 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:31.000037    4943 cpu_manager.go:414] "containerMap: removing container" podUID="f0e82596-0383-4ec9-a856-a6234e9da808" containerName="istio-proxy"
Oct 30 13:21:31.000175 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:31.000042    4943 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f0e82596-0383-4ec9-a856-a6234e9da808" containerName="istio-proxy"
Oct 30 13:21:31.000175 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:31.000052    4943 cpu_manager.go:414] "containerMap: removing container" podUID="bd83c13a-8f22-4ada-a894-90cd49e87782" containerName="istio-proxy"
Oct 30 13:21:31.000175 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:31.000058    4943 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="bd83c13a-8f22-4ada-a894-90cd49e87782" containerName="istio-proxy"
Oct 30 13:21:31.000175 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:31.000064    4943 cpu_manager.go:414] "containerMap: removing container" podUID="258db5e7-aa22-4bb6-b8ee-844241ca5399" containerName="istio-proxy"
Oct 30 13:21:31.000175 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:31.000069    4943 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="258db5e7-aa22-4bb6-b8ee-844241ca5399" containerName="istio-proxy"
Oct 30 13:21:31.000175 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:31.000134    4943 memory_manager.go:365] "RemoveStaleState removing state" podUID="f0ab36b0-4ba8-41da-8fc2-6a70d1082599" containerName="istio-proxy"
Oct 30 13:21:31.000175 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:31.000145    4943 memory_manager.go:365] "RemoveStaleState removing state" podUID="1c74e310-6638-4b15-97d8-65854946086a" containerName="istio-proxy"
Oct 30 13:21:31.000175 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:31.000151    4943 memory_manager.go:365] "RemoveStaleState removing state" podUID="80c84ecd-0a11-4d8e-ab46-2ba9efd4e676" containerName="payload-processing"
Oct 30 13:21:31.000175 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:31.000157    4943 memory_manager.go:365] "RemoveStaleState removing state" podUID="f0e82596-0383-4ec9-a856-a6234e9da808" containerName="istio-proxy"
Oct 30 13:21:31.000175 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:31.000163    4943 memory_manager.go:365] "RemoveStaleState removing state" podUID="bd83c13a-8f22-4ada-a894-90cd49e87782" containerName="istio-proxy"
Oct 30 13:21:31.000175 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:31.000169    4943 memory_manager.go:365] "RemoveStaleState removing state" podUID="258db5e7-aa22-4bb6-b8ee-844241ca5399" containerName="istio-proxy"
Oct 30 13:21:31.000175 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:31.000175    4943 memory_manager.go:365] "RemoveStaleState removing state" podUID="87aca420-1995-4b47-a522-0d60b607db07" containerName="revoke-keys"
Oct 30 13:21:31.005565 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:31.005545    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-6631f3dc-vg8sr"
Oct 30 13:21:31.035753 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:31.035725    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-6631f3dc-vg8sr"]
Oct 30 13:21:31.042478 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:31.042451    4943 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 13:21:31.042478 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:31.042477    4943 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Oct 30 13:21:31.042624 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:31.042452    4943 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Oct 30 13:21:31.042697 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:31.042674    4943 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-p9z57\""
Oct 30 13:21:31.071295 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:31.071242    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/92a51875-955c-40bd-994e-9ab0e8f46938-maas-api-xks-ca\") pod \"maas-api-revoke-sub-6631f3dc-vg8sr\" (UID: \"92a51875-955c-40bd-994e-9ab0e8f46938\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-6631f3dc-vg8sr"
Oct 30 13:21:31.071473 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:31.071450    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mq6cs\" (UniqueName: \"kubernetes.io/projected/92a51875-955c-40bd-994e-9ab0e8f46938-kube-api-access-mq6cs\") pod \"maas-api-revoke-sub-6631f3dc-vg8sr\" (UID: \"92a51875-955c-40bd-994e-9ab0e8f46938\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-6631f3dc-vg8sr"
Oct 30 13:21:31.071540 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:31.071517    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/92a51875-955c-40bd-994e-9ab0e8f46938-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-6631f3dc-vg8sr\" (UID: \"92a51875-955c-40bd-994e-9ab0e8f46938\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-6631f3dc-vg8sr"
Oct 30 13:21:31.172697 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:31.172669    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mq6cs\" (UniqueName: \"kubernetes.io/projected/92a51875-955c-40bd-994e-9ab0e8f46938-kube-api-access-mq6cs\") pod \"maas-api-revoke-sub-6631f3dc-vg8sr\" (UID: \"92a51875-955c-40bd-994e-9ab0e8f46938\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-6631f3dc-vg8sr"
Oct 30 13:21:31.172830 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:31.172713    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/92a51875-955c-40bd-994e-9ab0e8f46938-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-6631f3dc-vg8sr\" (UID: \"92a51875-955c-40bd-994e-9ab0e8f46938\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-6631f3dc-vg8sr"
Oct 30 13:21:31.172830 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:31.172760    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/92a51875-955c-40bd-994e-9ab0e8f46938-maas-api-xks-ca\") pod \"maas-api-revoke-sub-6631f3dc-vg8sr\" (UID: \"92a51875-955c-40bd-994e-9ab0e8f46938\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-6631f3dc-vg8sr"
Oct 30 13:21:31.173395 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:31.173373    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/92a51875-955c-40bd-994e-9ab0e8f46938-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-6631f3dc-vg8sr\" (UID: \"92a51875-955c-40bd-994e-9ab0e8f46938\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-6631f3dc-vg8sr"
Oct 30 13:21:31.175110 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:31.175089    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/92a51875-955c-40bd-994e-9ab0e8f46938-maas-api-xks-ca\") pod \"maas-api-revoke-sub-6631f3dc-vg8sr\" (UID: \"92a51875-955c-40bd-994e-9ab0e8f46938\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-6631f3dc-vg8sr"
Oct 30 13:21:31.199634 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:31.199601    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mq6cs\" (UniqueName: \"kubernetes.io/projected/92a51875-955c-40bd-994e-9ab0e8f46938-kube-api-access-mq6cs\") pod \"maas-api-revoke-sub-6631f3dc-vg8sr\" (UID: \"92a51875-955c-40bd-994e-9ab0e8f46938\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-6631f3dc-vg8sr"
Oct 30 13:21:31.319396 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:31.319324    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-6631f3dc-vg8sr"
Oct 30 13:21:31.628895 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:31.628867    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-6631f3dc-vg8sr"]
Oct 30 13:21:31.628895 ip-10-0-142-84 kubenswrapper[4943]: W1030 13:21:31.628880    4943 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod92a51875_955c_40bd_994e_9ab0e8f46938.slice/crio-3a13ecacd91d28527effa131589ed07ac839cd892a50eebe9612d5931b2923d3 WatchSource:0}: Error finding container 3a13ecacd91d28527effa131589ed07ac839cd892a50eebe9612d5931b2923d3: Status 404 returned error can't find the container with id 3a13ecacd91d28527effa131589ed07ac839cd892a50eebe9612d5931b2923d3
Oct 30 13:21:31.984552 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:31.984518    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-6631f3dc-vg8sr" event={"ID":"92a51875-955c-40bd-994e-9ab0e8f46938","Type":"ContainerStarted","Data":"9053a006ca5e71bbf11cd6aaa0c2198fd0bcf4b2ae2f2f356e3835f32c1bd754"}
Oct 30 13:21:31.984642 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:31.984581    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-6631f3dc-vg8sr" event={"ID":"92a51875-955c-40bd-994e-9ab0e8f46938","Type":"ContainerStarted","Data":"3a13ecacd91d28527effa131589ed07ac839cd892a50eebe9612d5931b2923d3"}
Oct 30 13:21:32.045124 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:32.045080    4943 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-6631f3dc-vg8sr" podStartSLOduration=2.045068235 podStartE2EDuration="2.045068235s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:21:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:21:32.042923715 +0000 UTC m=+1467.830927446" watchObservedRunningTime="2026-10-30 13:21:32.045068235 +0000 UTC m=+1467.833071962"
Oct 30 13:21:32.994845 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:32.994813    4943 generic.go:356] "Generic (PLEG): container finished" podID="92a51875-955c-40bd-994e-9ab0e8f46938" containerID="9053a006ca5e71bbf11cd6aaa0c2198fd0bcf4b2ae2f2f356e3835f32c1bd754" exitCode=0
Oct 30 13:21:32.995217 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:32.994903    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-6631f3dc-vg8sr" event={"ID":"92a51875-955c-40bd-994e-9ab0e8f46938","Type":"ContainerDied","Data":"9053a006ca5e71bbf11cd6aaa0c2198fd0bcf4b2ae2f2f356e3835f32c1bd754"}
Oct 30 13:21:34.152877 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:34.152847    4943 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-6631f3dc-vg8sr"
Oct 30 13:21:34.202906 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:34.202874    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/92a51875-955c-40bd-994e-9ab0e8f46938-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/92a51875-955c-40bd-994e-9ab0e8f46938-maas-api-openshift-ca\") pod \"92a51875-955c-40bd-994e-9ab0e8f46938\" (UID: \"92a51875-955c-40bd-994e-9ab0e8f46938\") "
Oct 30 13:21:34.203055 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:34.202917    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/92a51875-955c-40bd-994e-9ab0e8f46938-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/92a51875-955c-40bd-994e-9ab0e8f46938-maas-api-xks-ca\") pod \"92a51875-955c-40bd-994e-9ab0e8f46938\" (UID: \"92a51875-955c-40bd-994e-9ab0e8f46938\") "
Oct 30 13:21:34.203055 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:34.203002    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/92a51875-955c-40bd-994e-9ab0e8f46938-kube-api-access-mq6cs\" (UniqueName: \"kubernetes.io/projected/92a51875-955c-40bd-994e-9ab0e8f46938-kube-api-access-mq6cs\") pod \"92a51875-955c-40bd-994e-9ab0e8f46938\" (UID: \"92a51875-955c-40bd-994e-9ab0e8f46938\") "
Oct 30 13:21:34.203353 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:34.203276    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/92a51875-955c-40bd-994e-9ab0e8f46938-maas-api-openshift-ca" pod "92a51875-955c-40bd-994e-9ab0e8f46938" (UID: "92a51875-955c-40bd-994e-9ab0e8f46938"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 13:21:34.203442 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:34.203422    4943 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/92a51875-955c-40bd-994e-9ab0e8f46938-maas-api-openshift-ca\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:21:34.217853 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:34.217822    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/92a51875-955c-40bd-994e-9ab0e8f46938-kube-api-access-mq6cs" pod "92a51875-955c-40bd-994e-9ab0e8f46938" (UID: "92a51875-955c-40bd-994e-9ab0e8f46938"). InnerVolumeSpecName "kube-api-access-mq6cs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:21:34.225428 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:34.225371    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/92a51875-955c-40bd-994e-9ab0e8f46938-maas-api-xks-ca" pod "92a51875-955c-40bd-994e-9ab0e8f46938" (UID: "92a51875-955c-40bd-994e-9ab0e8f46938"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 13:21:34.310447 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:34.310342    4943 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/92a51875-955c-40bd-994e-9ab0e8f46938-maas-api-xks-ca\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:21:34.310447 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:34.310389    4943 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mq6cs\" (UniqueName: \"kubernetes.io/projected/92a51875-955c-40bd-994e-9ab0e8f46938-kube-api-access-mq6cs\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:21:35.011111 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:35.011082    4943 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-6631f3dc-vg8sr"
Oct 30 13:21:35.011111 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:35.011072    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-6631f3dc-vg8sr" event={"ID":"92a51875-955c-40bd-994e-9ab0e8f46938","Type":"ContainerDied","Data":"3a13ecacd91d28527effa131589ed07ac839cd892a50eebe9612d5931b2923d3"}
Oct 30 13:21:35.011373 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:35.011156    4943 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3a13ecacd91d28527effa131589ed07ac839cd892a50eebe9612d5931b2923d3"
Oct 30 13:21:35.199948 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:35.199892    4943 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-ab3176-5f84755579-6t85j" podUID="1d807dba-b7c2-412d-8be6-aa6a4f6db610" containerName="payload-pre-processing" containerID="cri-o://f9d2a89a5d0a684bffda22e5c8c2deb921f0b76ad3737c2dcc6ea495660b3321" gracePeriod=25
Oct 30 13:21:35.955100 ip-10-0-142-84 kubenswrapper[4943]: E1030 13:21:35.955062    4943 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-knqz6" containerName="extractor"
Oct 30 13:21:36.283384 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:36.283304    4943 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-qzz29"]
Oct 30 13:21:36.283746 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:36.283551    4943 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-qzz29" podUID="7bae0af9-fbeb-4480-9452-1abab86550e8" containerName="istio-proxy" containerID="cri-o://66b125e7d65ecd902d1fc405dc5610d87fe7ddca1c374f5c597feab53dd0dba7" gracePeriod=30
Oct 30 13:21:36.297063 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:36.297030    4943 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-vqzsp"]
Oct 30 13:21:36.297271 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:36.297229    4943 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-vqzsp" podUID="6b9fc86d-7710-41b3-8d41-35cb717eb50e" containerName="istio-proxy" containerID="cri-o://8db59a7efd7b42cde501a97b371d8429e86a1f57f77b62ed00959cb067a43748" gracePeriod=30
Oct 30 13:21:36.300450 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:36.300429    4943 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-586tt"]
Oct 30 13:21:36.300667 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:36.300628    4943 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-586tt" podUID="9c5b6db1-55db-447c-a3a9-2e24a9dc9dc6" containerName="istio-proxy" containerID="cri-o://ec6a6067cc5996f4434d40951dca64d038e1e6b22da9fd8cb27397b56344a9d9" gracePeriod=30
Oct 30 13:21:37.996475 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:37.996441    4943 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-ab3176-5f84755579-6t85j" podUID="1d807dba-b7c2-412d-8be6-aa6a4f6db610" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 13:21:39.473059 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:39.473000    4943 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w6-ab3176/simulator-w6-ab3176-kserve-6d9b64549f-dc8xz" podUID="da306792-4f80-4df4-aa44-82991ab856e4" containerName="main" containerID="cri-o://fd93c8c8b8d5bcd559d09f59b456cbaf348baf2bb7739c60ef28ae03c812bc62" gracePeriod=45
Oct 30 13:21:39.724289 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:39.724224    4943 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-ab3176/simulator-w6-ab3176-kserve-6d9b64549f-dc8xz"
Oct 30 13:21:39.863283 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:39.863225    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/da306792-4f80-4df4-aa44-82991ab856e4-kube-api-access-48mt6\" (UniqueName: \"kubernetes.io/projected/da306792-4f80-4df4-aa44-82991ab856e4-kube-api-access-48mt6\") pod \"da306792-4f80-4df4-aa44-82991ab856e4\" (UID: \"da306792-4f80-4df4-aa44-82991ab856e4\") "
Oct 30 13:21:39.863471 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:39.863302    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/da306792-4f80-4df4-aa44-82991ab856e4-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/da306792-4f80-4df4-aa44-82991ab856e4-model-cache\") pod \"da306792-4f80-4df4-aa44-82991ab856e4\" (UID: \"da306792-4f80-4df4-aa44-82991ab856e4\") "
Oct 30 13:21:39.863471 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:39.863346    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/da306792-4f80-4df4-aa44-82991ab856e4-home\" (UniqueName: \"kubernetes.io/empty-dir/da306792-4f80-4df4-aa44-82991ab856e4-home\") pod \"da306792-4f80-4df4-aa44-82991ab856e4\" (UID: \"da306792-4f80-4df4-aa44-82991ab856e4\") "
Oct 30 13:21:39.863471 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:39.863383    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/da306792-4f80-4df4-aa44-82991ab856e4-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/da306792-4f80-4df4-aa44-82991ab856e4-tmp-dir\") pod \"da306792-4f80-4df4-aa44-82991ab856e4\" (UID: \"da306792-4f80-4df4-aa44-82991ab856e4\") "
Oct 30 13:21:39.863471 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:39.863423    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/da306792-4f80-4df4-aa44-82991ab856e4-tls-certs\" (UniqueName: \"kubernetes.io/secret/da306792-4f80-4df4-aa44-82991ab856e4-tls-certs\") pod \"da306792-4f80-4df4-aa44-82991ab856e4\" (UID: \"da306792-4f80-4df4-aa44-82991ab856e4\") "
Oct 30 13:21:39.863471 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:39.863466    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/da306792-4f80-4df4-aa44-82991ab856e4-dshm\" (UniqueName: \"kubernetes.io/empty-dir/da306792-4f80-4df4-aa44-82991ab856e4-dshm\") pod \"da306792-4f80-4df4-aa44-82991ab856e4\" (UID: \"da306792-4f80-4df4-aa44-82991ab856e4\") "
Oct 30 13:21:39.863731 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:39.863604    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/da306792-4f80-4df4-aa44-82991ab856e4-home" pod "da306792-4f80-4df4-aa44-82991ab856e4" (UID: "da306792-4f80-4df4-aa44-82991ab856e4"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:21:39.863731 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:39.863609    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/da306792-4f80-4df4-aa44-82991ab856e4-model-cache" pod "da306792-4f80-4df4-aa44-82991ab856e4" (UID: "da306792-4f80-4df4-aa44-82991ab856e4"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:21:39.863731 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:39.863621    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/da306792-4f80-4df4-aa44-82991ab856e4-tmp-dir" pod "da306792-4f80-4df4-aa44-82991ab856e4" (UID: "da306792-4f80-4df4-aa44-82991ab856e4"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:21:39.863831 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:39.863803    4943 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/da306792-4f80-4df4-aa44-82991ab856e4-model-cache\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:21:39.863831 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:39.863824    4943 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/da306792-4f80-4df4-aa44-82991ab856e4-home\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:21:39.863898 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:39.863861    4943 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/da306792-4f80-4df4-aa44-82991ab856e4-tmp-dir\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:21:39.865514 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:39.865491    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/da306792-4f80-4df4-aa44-82991ab856e4-kube-api-access-48mt6" pod "da306792-4f80-4df4-aa44-82991ab856e4" (UID: "da306792-4f80-4df4-aa44-82991ab856e4"). InnerVolumeSpecName "kube-api-access-48mt6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:21:39.865595 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:39.865566    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/da306792-4f80-4df4-aa44-82991ab856e4-dshm" pod "da306792-4f80-4df4-aa44-82991ab856e4" (UID: "da306792-4f80-4df4-aa44-82991ab856e4"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:21:39.865667 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:39.865648    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/da306792-4f80-4df4-aa44-82991ab856e4-tls-certs" pod "da306792-4f80-4df4-aa44-82991ab856e4" (UID: "da306792-4f80-4df4-aa44-82991ab856e4"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 13:21:39.965176 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:39.965142    4943 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-48mt6\" (UniqueName: \"kubernetes.io/projected/da306792-4f80-4df4-aa44-82991ab856e4-kube-api-access-48mt6\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:21:39.965176 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:39.965173    4943 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/da306792-4f80-4df4-aa44-82991ab856e4-tls-certs\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:21:39.965176 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:39.965184    4943 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/da306792-4f80-4df4-aa44-82991ab856e4-dshm\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:21:40.047510 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:40.047435    4943 generic.go:356] "Generic (PLEG): container finished" podID="da306792-4f80-4df4-aa44-82991ab856e4" containerID="fd93c8c8b8d5bcd559d09f59b456cbaf348baf2bb7739c60ef28ae03c812bc62" exitCode=0
Oct 30 13:21:40.047510 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:40.047481    4943 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-ab3176/simulator-w6-ab3176-kserve-6d9b64549f-dc8xz"
Oct 30 13:21:40.047706 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:40.047516    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-ab3176/simulator-w6-ab3176-kserve-6d9b64549f-dc8xz" event={"ID":"da306792-4f80-4df4-aa44-82991ab856e4","Type":"ContainerDied","Data":"fd93c8c8b8d5bcd559d09f59b456cbaf348baf2bb7739c60ef28ae03c812bc62"}
Oct 30 13:21:40.047706 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:40.047561    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-ab3176/simulator-w6-ab3176-kserve-6d9b64549f-dc8xz" event={"ID":"da306792-4f80-4df4-aa44-82991ab856e4","Type":"ContainerDied","Data":"0ba6be2b1bb09e4a45ed6b724e833540fce2c9c2ffd2501fd75c47bd6afcb4e8"}
Oct 30 13:21:40.047706 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:40.047579    4943 scope.go:122] "RemoveContainer" containerID="fd93c8c8b8d5bcd559d09f59b456cbaf348baf2bb7739c60ef28ae03c812bc62"
Oct 30 13:21:40.059040 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:40.059022    4943 scope.go:122] "RemoveContainer" containerID="fd93c8c8b8d5bcd559d09f59b456cbaf348baf2bb7739c60ef28ae03c812bc62"
Oct 30 13:21:40.059343 ip-10-0-142-84 kubenswrapper[4943]: E1030 13:21:40.059324    4943 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fd93c8c8b8d5bcd559d09f59b456cbaf348baf2bb7739c60ef28ae03c812bc62\": container with ID starting with fd93c8c8b8d5bcd559d09f59b456cbaf348baf2bb7739c60ef28ae03c812bc62 not found: ID does not exist" containerID="fd93c8c8b8d5bcd559d09f59b456cbaf348baf2bb7739c60ef28ae03c812bc62"
Oct 30 13:21:40.059388 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:40.059352    4943 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fd93c8c8b8d5bcd559d09f59b456cbaf348baf2bb7739c60ef28ae03c812bc62"} err="failed to get container status \"fd93c8c8b8d5bcd559d09f59b456cbaf348baf2bb7739c60ef28ae03c812bc62\": rpc error: code = NotFound desc = could not find container \"fd93c8c8b8d5bcd559d09f59b456cbaf348baf2bb7739c60ef28ae03c812bc62\": container with ID starting with fd93c8c8b8d5bcd559d09f59b456cbaf348baf2bb7739c60ef28ae03c812bc62 not found: ID does not exist"
Oct 30 13:21:40.108465 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:40.108441    4943 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w6-ab3176/simulator-w6-ab3176-kserve-6d9b64549f-dc8xz"]
Oct 30 13:21:40.122133 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:40.122099    4943 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w6-ab3176/simulator-w6-ab3176-kserve-6d9b64549f-dc8xz"]
Oct 30 13:21:40.350490 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:40.350404    4943 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="da306792-4f80-4df4-aa44-82991ab856e4" path="/var/lib/kubelet/pods/da306792-4f80-4df4-aa44-82991ab856e4/volumes"
Oct 30 13:21:41.572923 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:41.572783    4943 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-qzz29"
Oct 30 13:21:41.596779 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:41.596758    4943 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-vqzsp"
Oct 30 13:21:41.622559 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:41.622535    4943 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-586tt"
Oct 30 13:21:41.681375 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:41.681346    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9c5b6db1-55db-447c-a3a9-2e24a9dc9dc6-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9c5b6db1-55db-447c-a3a9-2e24a9dc9dc6-credential-socket\") pod \"9c5b6db1-55db-447c-a3a9-2e24a9dc9dc6\" (UID: \"9c5b6db1-55db-447c-a3a9-2e24a9dc9dc6\") "
Oct 30 13:21:41.681521 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:41.681387    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6b9fc86d-7710-41b3-8d41-35cb717eb50e-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6b9fc86d-7710-41b3-8d41-35cb717eb50e-credential-socket\") pod \"6b9fc86d-7710-41b3-8d41-35cb717eb50e\" (UID: \"6b9fc86d-7710-41b3-8d41-35cb717eb50e\") "
Oct 30 13:21:41.681521 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:41.681436    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/9c5b6db1-55db-447c-a3a9-2e24a9dc9dc6-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9c5b6db1-55db-447c-a3a9-2e24a9dc9dc6-istiod-ca-cert\") pod \"9c5b6db1-55db-447c-a3a9-2e24a9dc9dc6\" (UID: \"9c5b6db1-55db-447c-a3a9-2e24a9dc9dc6\") "
Oct 30 13:21:41.681521 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:41.681466    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7bae0af9-fbeb-4480-9452-1abab86550e8-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7bae0af9-fbeb-4480-9452-1abab86550e8-istio-data\") pod \"7bae0af9-fbeb-4480-9452-1abab86550e8\" (UID: \"7bae0af9-fbeb-4480-9452-1abab86550e8\") "
Oct 30 13:21:41.681521 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:41.681497    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9c5b6db1-55db-447c-a3a9-2e24a9dc9dc6-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9c5b6db1-55db-447c-a3a9-2e24a9dc9dc6-istio-data\") pod \"9c5b6db1-55db-447c-a3a9-2e24a9dc9dc6\" (UID: \"9c5b6db1-55db-447c-a3a9-2e24a9dc9dc6\") "
Oct 30 13:21:41.681758 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:41.681534    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6b9fc86d-7710-41b3-8d41-35cb717eb50e-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6b9fc86d-7710-41b3-8d41-35cb717eb50e-istio-data\") pod \"6b9fc86d-7710-41b3-8d41-35cb717eb50e\" (UID: \"6b9fc86d-7710-41b3-8d41-35cb717eb50e\") "
Oct 30 13:21:41.681758 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:41.681581    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9c5b6db1-55db-447c-a3a9-2e24a9dc9dc6-kube-api-access-92npq\" (UniqueName: \"kubernetes.io/projected/9c5b6db1-55db-447c-a3a9-2e24a9dc9dc6-kube-api-access-92npq\") pod \"9c5b6db1-55db-447c-a3a9-2e24a9dc9dc6\" (UID: \"9c5b6db1-55db-447c-a3a9-2e24a9dc9dc6\") "
Oct 30 13:21:41.681758 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:41.681633    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/7bae0af9-fbeb-4480-9452-1abab86550e8-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7bae0af9-fbeb-4480-9452-1abab86550e8-istio-podinfo\") pod \"7bae0af9-fbeb-4480-9452-1abab86550e8\" (UID: \"7bae0af9-fbeb-4480-9452-1abab86550e8\") "
Oct 30 13:21:41.681758 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:41.681650    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9c5b6db1-55db-447c-a3a9-2e24a9dc9dc6-credential-socket" pod "9c5b6db1-55db-447c-a3a9-2e24a9dc9dc6" (UID: "9c5b6db1-55db-447c-a3a9-2e24a9dc9dc6"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:21:41.681758 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:41.681661    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7bae0af9-fbeb-4480-9452-1abab86550e8-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7bae0af9-fbeb-4480-9452-1abab86550e8-workload-socket\") pod \"7bae0af9-fbeb-4480-9452-1abab86550e8\" (UID: \"7bae0af9-fbeb-4480-9452-1abab86550e8\") "
Oct 30 13:21:41.681758 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:41.681698    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7bae0af9-fbeb-4480-9452-1abab86550e8-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7bae0af9-fbeb-4480-9452-1abab86550e8-credential-socket\") pod \"7bae0af9-fbeb-4480-9452-1abab86550e8\" (UID: \"7bae0af9-fbeb-4480-9452-1abab86550e8\") "
Oct 30 13:21:41.681758 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:41.681729    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6b9fc86d-7710-41b3-8d41-35cb717eb50e-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6b9fc86d-7710-41b3-8d41-35cb717eb50e-workload-certs\") pod \"6b9fc86d-7710-41b3-8d41-35cb717eb50e\" (UID: \"6b9fc86d-7710-41b3-8d41-35cb717eb50e\") "
Oct 30 13:21:41.682106 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:41.681766    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6b9fc86d-7710-41b3-8d41-35cb717eb50e-istio-token\" (UniqueName: \"kubernetes.io/projected/6b9fc86d-7710-41b3-8d41-35cb717eb50e-istio-token\") pod \"6b9fc86d-7710-41b3-8d41-35cb717eb50e\" (UID: \"6b9fc86d-7710-41b3-8d41-35cb717eb50e\") "
Oct 30 13:21:41.682106 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:41.681806    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7bae0af9-fbeb-4480-9452-1abab86550e8-kube-api-access-jdh6c\" (UniqueName: \"kubernetes.io/projected/7bae0af9-fbeb-4480-9452-1abab86550e8-kube-api-access-jdh6c\") pod \"7bae0af9-fbeb-4480-9452-1abab86550e8\" (UID: \"7bae0af9-fbeb-4480-9452-1abab86550e8\") "
Oct 30 13:21:41.682106 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:41.681842    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6b9fc86d-7710-41b3-8d41-35cb717eb50e-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6b9fc86d-7710-41b3-8d41-35cb717eb50e-istio-envoy\") pod \"6b9fc86d-7710-41b3-8d41-35cb717eb50e\" (UID: \"6b9fc86d-7710-41b3-8d41-35cb717eb50e\") "
Oct 30 13:21:41.682106 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:41.681874    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9c5b6db1-55db-447c-a3a9-2e24a9dc9dc6-istio-token\" (UniqueName: \"kubernetes.io/projected/9c5b6db1-55db-447c-a3a9-2e24a9dc9dc6-istio-token\") pod \"9c5b6db1-55db-447c-a3a9-2e24a9dc9dc6\" (UID: \"9c5b6db1-55db-447c-a3a9-2e24a9dc9dc6\") "
Oct 30 13:21:41.682106 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:41.681923    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6b9fc86d-7710-41b3-8d41-35cb717eb50e-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6b9fc86d-7710-41b3-8d41-35cb717eb50e-istiod-ca-cert\") pod \"6b9fc86d-7710-41b3-8d41-35cb717eb50e\" (UID: \"6b9fc86d-7710-41b3-8d41-35cb717eb50e\") "
Oct 30 13:21:41.682106 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:41.681955    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6b9fc86d-7710-41b3-8d41-35cb717eb50e-kube-api-access-cz4jn\" (UniqueName: \"kubernetes.io/projected/6b9fc86d-7710-41b3-8d41-35cb717eb50e-kube-api-access-cz4jn\") pod \"6b9fc86d-7710-41b3-8d41-35cb717eb50e\" (UID: \"6b9fc86d-7710-41b3-8d41-35cb717eb50e\") "
Oct 30 13:21:41.682106 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:41.681989    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7bae0af9-fbeb-4480-9452-1abab86550e8-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7bae0af9-fbeb-4480-9452-1abab86550e8-workload-certs\") pod \"7bae0af9-fbeb-4480-9452-1abab86550e8\" (UID: \"7bae0af9-fbeb-4480-9452-1abab86550e8\") "
Oct 30 13:21:41.682106 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:41.682022    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/7bae0af9-fbeb-4480-9452-1abab86550e8-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7bae0af9-fbeb-4480-9452-1abab86550e8-istiod-ca-cert\") pod \"7bae0af9-fbeb-4480-9452-1abab86550e8\" (UID: \"7bae0af9-fbeb-4480-9452-1abab86550e8\") "
Oct 30 13:21:41.682106 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:41.682049    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9c5b6db1-55db-447c-a3a9-2e24a9dc9dc6-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9c5b6db1-55db-447c-a3a9-2e24a9dc9dc6-workload-socket\") pod \"9c5b6db1-55db-447c-a3a9-2e24a9dc9dc6\" (UID: \"9c5b6db1-55db-447c-a3a9-2e24a9dc9dc6\") "
Oct 30 13:21:41.682616 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:41.682082    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/9c5b6db1-55db-447c-a3a9-2e24a9dc9dc6-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9c5b6db1-55db-447c-a3a9-2e24a9dc9dc6-istio-podinfo\") pod \"9c5b6db1-55db-447c-a3a9-2e24a9dc9dc6\" (UID: \"9c5b6db1-55db-447c-a3a9-2e24a9dc9dc6\") "
Oct 30 13:21:41.682616 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:41.682173    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9c5b6db1-55db-447c-a3a9-2e24a9dc9dc6-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9c5b6db1-55db-447c-a3a9-2e24a9dc9dc6-workload-certs\") pod \"9c5b6db1-55db-447c-a3a9-2e24a9dc9dc6\" (UID: \"9c5b6db1-55db-447c-a3a9-2e24a9dc9dc6\") "
Oct 30 13:21:41.682616 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:41.682216    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6b9fc86d-7710-41b3-8d41-35cb717eb50e-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6b9fc86d-7710-41b3-8d41-35cb717eb50e-workload-socket\") pod \"6b9fc86d-7710-41b3-8d41-35cb717eb50e\" (UID: \"6b9fc86d-7710-41b3-8d41-35cb717eb50e\") "
Oct 30 13:21:41.682616 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:41.682293    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7bae0af9-fbeb-4480-9452-1abab86550e8-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7bae0af9-fbeb-4480-9452-1abab86550e8-istio-envoy\") pod \"7bae0af9-fbeb-4480-9452-1abab86550e8\" (UID: \"7bae0af9-fbeb-4480-9452-1abab86550e8\") "
Oct 30 13:21:41.682616 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:41.682330    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7bae0af9-fbeb-4480-9452-1abab86550e8-istio-token\" (UniqueName: \"kubernetes.io/projected/7bae0af9-fbeb-4480-9452-1abab86550e8-istio-token\") pod \"7bae0af9-fbeb-4480-9452-1abab86550e8\" (UID: \"7bae0af9-fbeb-4480-9452-1abab86550e8\") "
Oct 30 13:21:41.682616 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:41.682377    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9c5b6db1-55db-447c-a3a9-2e24a9dc9dc6-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9c5b6db1-55db-447c-a3a9-2e24a9dc9dc6-istio-envoy\") pod \"9c5b6db1-55db-447c-a3a9-2e24a9dc9dc6\" (UID: \"9c5b6db1-55db-447c-a3a9-2e24a9dc9dc6\") "
Oct 30 13:21:41.682616 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:41.682410    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/6b9fc86d-7710-41b3-8d41-35cb717eb50e-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6b9fc86d-7710-41b3-8d41-35cb717eb50e-istio-podinfo\") pod \"6b9fc86d-7710-41b3-8d41-35cb717eb50e\" (UID: \"6b9fc86d-7710-41b3-8d41-35cb717eb50e\") "
Oct 30 13:21:41.682975 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:41.682810    4943 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9c5b6db1-55db-447c-a3a9-2e24a9dc9dc6-credential-socket\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:21:41.683790 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:41.681843    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6b9fc86d-7710-41b3-8d41-35cb717eb50e-istio-data" pod "6b9fc86d-7710-41b3-8d41-35cb717eb50e" (UID: "6b9fc86d-7710-41b3-8d41-35cb717eb50e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:21:41.683790 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:41.682009    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7bae0af9-fbeb-4480-9452-1abab86550e8-credential-socket" pod "7bae0af9-fbeb-4480-9452-1abab86550e8" (UID: "7bae0af9-fbeb-4480-9452-1abab86550e8"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:21:41.683790 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:41.682557    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7bae0af9-fbeb-4480-9452-1abab86550e8-istio-data" pod "7bae0af9-fbeb-4480-9452-1abab86550e8" (UID: "7bae0af9-fbeb-4480-9452-1abab86550e8"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:21:41.683790 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:41.682667    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9c5b6db1-55db-447c-a3a9-2e24a9dc9dc6-istio-data" pod "9c5b6db1-55db-447c-a3a9-2e24a9dc9dc6" (UID: "9c5b6db1-55db-447c-a3a9-2e24a9dc9dc6"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:21:41.683790 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:41.682752    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9c5b6db1-55db-447c-a3a9-2e24a9dc9dc6-istiod-ca-cert" pod "9c5b6db1-55db-447c-a3a9-2e24a9dc9dc6" (UID: "9c5b6db1-55db-447c-a3a9-2e24a9dc9dc6"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 13:21:41.683790 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:41.683106    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7bae0af9-fbeb-4480-9452-1abab86550e8-istiod-ca-cert" pod "7bae0af9-fbeb-4480-9452-1abab86550e8" (UID: "7bae0af9-fbeb-4480-9452-1abab86550e8"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 13:21:41.683790 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:41.683318    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6b9fc86d-7710-41b3-8d41-35cb717eb50e-workload-certs" pod "6b9fc86d-7710-41b3-8d41-35cb717eb50e" (UID: "6b9fc86d-7710-41b3-8d41-35cb717eb50e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:21:41.683790 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:41.683386    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6b9fc86d-7710-41b3-8d41-35cb717eb50e-istiod-ca-cert" pod "6b9fc86d-7710-41b3-8d41-35cb717eb50e" (UID: "6b9fc86d-7710-41b3-8d41-35cb717eb50e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 13:21:41.683790 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:41.683742    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6b9fc86d-7710-41b3-8d41-35cb717eb50e-credential-socket" pod "6b9fc86d-7710-41b3-8d41-35cb717eb50e" (UID: "6b9fc86d-7710-41b3-8d41-35cb717eb50e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:21:41.684480 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:41.684020    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6b9fc86d-7710-41b3-8d41-35cb717eb50e-workload-socket" pod "6b9fc86d-7710-41b3-8d41-35cb717eb50e" (UID: "6b9fc86d-7710-41b3-8d41-35cb717eb50e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:21:41.685880 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:41.685104    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7bae0af9-fbeb-4480-9452-1abab86550e8-workload-socket" pod "7bae0af9-fbeb-4480-9452-1abab86550e8" (UID: "7bae0af9-fbeb-4480-9452-1abab86550e8"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:21:41.685880 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:41.685571    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9c5b6db1-55db-447c-a3a9-2e24a9dc9dc6-workload-socket" pod "9c5b6db1-55db-447c-a3a9-2e24a9dc9dc6" (UID: "9c5b6db1-55db-447c-a3a9-2e24a9dc9dc6"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:21:41.685880 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:41.685629    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9c5b6db1-55db-447c-a3a9-2e24a9dc9dc6-workload-certs" pod "9c5b6db1-55db-447c-a3a9-2e24a9dc9dc6" (UID: "9c5b6db1-55db-447c-a3a9-2e24a9dc9dc6"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:21:41.686228 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:41.686200    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/6b9fc86d-7710-41b3-8d41-35cb717eb50e-istio-podinfo" pod "6b9fc86d-7710-41b3-8d41-35cb717eb50e" (UID: "6b9fc86d-7710-41b3-8d41-35cb717eb50e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 13:21:41.686486 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:41.686196    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/7bae0af9-fbeb-4480-9452-1abab86550e8-istio-podinfo" pod "7bae0af9-fbeb-4480-9452-1abab86550e8" (UID: "7bae0af9-fbeb-4480-9452-1abab86550e8"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 13:21:41.686559 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:41.686534    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7bae0af9-fbeb-4480-9452-1abab86550e8-workload-certs" pod "7bae0af9-fbeb-4480-9452-1abab86550e8" (UID: "7bae0af9-fbeb-4480-9452-1abab86550e8"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:21:41.689695 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:41.689668    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9c5b6db1-55db-447c-a3a9-2e24a9dc9dc6-istio-envoy" pod "9c5b6db1-55db-447c-a3a9-2e24a9dc9dc6" (UID: "9c5b6db1-55db-447c-a3a9-2e24a9dc9dc6"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:21:41.689695 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:41.689683    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6b9fc86d-7710-41b3-8d41-35cb717eb50e-kube-api-access-cz4jn" pod "6b9fc86d-7710-41b3-8d41-35cb717eb50e" (UID: "6b9fc86d-7710-41b3-8d41-35cb717eb50e"). InnerVolumeSpecName "kube-api-access-cz4jn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:21:41.689854 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:41.689798    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/9c5b6db1-55db-447c-a3a9-2e24a9dc9dc6-istio-podinfo" pod "9c5b6db1-55db-447c-a3a9-2e24a9dc9dc6" (UID: "9c5b6db1-55db-447c-a3a9-2e24a9dc9dc6"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 13:21:41.689854 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:41.689810    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9c5b6db1-55db-447c-a3a9-2e24a9dc9dc6-istio-token" pod "9c5b6db1-55db-447c-a3a9-2e24a9dc9dc6" (UID: "9c5b6db1-55db-447c-a3a9-2e24a9dc9dc6"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:21:41.689952 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:41.689874    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9c5b6db1-55db-447c-a3a9-2e24a9dc9dc6-kube-api-access-92npq" pod "9c5b6db1-55db-447c-a3a9-2e24a9dc9dc6" (UID: "9c5b6db1-55db-447c-a3a9-2e24a9dc9dc6"). InnerVolumeSpecName "kube-api-access-92npq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:21:41.690004 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:41.689975    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7bae0af9-fbeb-4480-9452-1abab86550e8-kube-api-access-jdh6c" pod "7bae0af9-fbeb-4480-9452-1abab86550e8" (UID: "7bae0af9-fbeb-4480-9452-1abab86550e8"). InnerVolumeSpecName "kube-api-access-jdh6c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:21:41.690050 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:41.689995    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7bae0af9-fbeb-4480-9452-1abab86550e8-istio-envoy" pod "7bae0af9-fbeb-4480-9452-1abab86550e8" (UID: "7bae0af9-fbeb-4480-9452-1abab86550e8"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:21:41.690050 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:41.690030    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7bae0af9-fbeb-4480-9452-1abab86550e8-istio-token" pod "7bae0af9-fbeb-4480-9452-1abab86550e8" (UID: "7bae0af9-fbeb-4480-9452-1abab86550e8"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:21:41.691632 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:41.691605    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6b9fc86d-7710-41b3-8d41-35cb717eb50e-istio-token" pod "6b9fc86d-7710-41b3-8d41-35cb717eb50e" (UID: "6b9fc86d-7710-41b3-8d41-35cb717eb50e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:21:41.692360 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:41.692336    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6b9fc86d-7710-41b3-8d41-35cb717eb50e-istio-envoy" pod "6b9fc86d-7710-41b3-8d41-35cb717eb50e" (UID: "6b9fc86d-7710-41b3-8d41-35cb717eb50e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:21:41.783916 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:41.783876    4943 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9c5b6db1-55db-447c-a3a9-2e24a9dc9dc6-istiod-ca-cert\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:21:41.783916 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:41.783909    4943 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7bae0af9-fbeb-4480-9452-1abab86550e8-istio-data\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:21:41.783916 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:41.783918    4943 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9c5b6db1-55db-447c-a3a9-2e24a9dc9dc6-istio-data\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:21:41.783916 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:41.783926    4943 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6b9fc86d-7710-41b3-8d41-35cb717eb50e-istio-data\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:21:41.784179 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:41.783935    4943 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-92npq\" (UniqueName: \"kubernetes.io/projected/9c5b6db1-55db-447c-a3a9-2e24a9dc9dc6-kube-api-access-92npq\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:21:41.784179 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:41.783944    4943 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7bae0af9-fbeb-4480-9452-1abab86550e8-istio-podinfo\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:21:41.784179 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:41.783953    4943 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7bae0af9-fbeb-4480-9452-1abab86550e8-workload-socket\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:21:41.784179 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:41.783962    4943 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7bae0af9-fbeb-4480-9452-1abab86550e8-credential-socket\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:21:41.784179 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:41.783971    4943 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6b9fc86d-7710-41b3-8d41-35cb717eb50e-workload-certs\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:21:41.784179 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:41.783979    4943 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6b9fc86d-7710-41b3-8d41-35cb717eb50e-istio-token\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:21:41.784179 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:41.783986    4943 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jdh6c\" (UniqueName: \"kubernetes.io/projected/7bae0af9-fbeb-4480-9452-1abab86550e8-kube-api-access-jdh6c\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:21:41.784179 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:41.783994    4943 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6b9fc86d-7710-41b3-8d41-35cb717eb50e-istio-envoy\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:21:41.784179 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:41.784002    4943 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9c5b6db1-55db-447c-a3a9-2e24a9dc9dc6-istio-token\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:21:41.784179 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:41.784011    4943 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6b9fc86d-7710-41b3-8d41-35cb717eb50e-istiod-ca-cert\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:21:41.784179 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:41.784019    4943 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cz4jn\" (UniqueName: \"kubernetes.io/projected/6b9fc86d-7710-41b3-8d41-35cb717eb50e-kube-api-access-cz4jn\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:21:41.784179 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:41.784028    4943 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7bae0af9-fbeb-4480-9452-1abab86550e8-workload-certs\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:21:41.784179 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:41.784038    4943 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7bae0af9-fbeb-4480-9452-1abab86550e8-istiod-ca-cert\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:21:41.784179 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:41.784046    4943 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9c5b6db1-55db-447c-a3a9-2e24a9dc9dc6-workload-socket\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:21:41.784179 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:41.784053    4943 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9c5b6db1-55db-447c-a3a9-2e24a9dc9dc6-istio-podinfo\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:21:41.784179 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:41.784061    4943 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9c5b6db1-55db-447c-a3a9-2e24a9dc9dc6-workload-certs\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:21:41.784179 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:41.784068    4943 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6b9fc86d-7710-41b3-8d41-35cb717eb50e-workload-socket\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:21:41.784179 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:41.784075    4943 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7bae0af9-fbeb-4480-9452-1abab86550e8-istio-envoy\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:21:41.784179 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:41.784085    4943 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7bae0af9-fbeb-4480-9452-1abab86550e8-istio-token\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:21:41.784179 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:41.784092    4943 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9c5b6db1-55db-447c-a3a9-2e24a9dc9dc6-istio-envoy\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:21:41.784179 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:41.784099    4943 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6b9fc86d-7710-41b3-8d41-35cb717eb50e-istio-podinfo\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:21:41.784179 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:41.784108    4943 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6b9fc86d-7710-41b3-8d41-35cb717eb50e-credential-socket\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:21:42.064937 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:42.064898    4943 generic.go:356] "Generic (PLEG): container finished" podID="9c5b6db1-55db-447c-a3a9-2e24a9dc9dc6" containerID="ec6a6067cc5996f4434d40951dca64d038e1e6b22da9fd8cb27397b56344a9d9" exitCode=0
Oct 30 13:21:42.065136 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:42.064958    4943 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-586tt"
Oct 30 13:21:42.065136 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:42.064982    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-586tt" event={"ID":"9c5b6db1-55db-447c-a3a9-2e24a9dc9dc6","Type":"ContainerDied","Data":"ec6a6067cc5996f4434d40951dca64d038e1e6b22da9fd8cb27397b56344a9d9"}
Oct 30 13:21:42.065136 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:42.065039    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-586tt" event={"ID":"9c5b6db1-55db-447c-a3a9-2e24a9dc9dc6","Type":"ContainerDied","Data":"24993988e17052f2727f3f8b1cafabeee52d1c253548cb432921cbd5915d4ead"}
Oct 30 13:21:42.065136 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:42.065065    4943 scope.go:122] "RemoveContainer" containerID="ec6a6067cc5996f4434d40951dca64d038e1e6b22da9fd8cb27397b56344a9d9"
Oct 30 13:21:42.066506 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:42.066480    4943 generic.go:356] "Generic (PLEG): container finished" podID="6b9fc86d-7710-41b3-8d41-35cb717eb50e" containerID="8db59a7efd7b42cde501a97b371d8429e86a1f57f77b62ed00959cb067a43748" exitCode=0
Oct 30 13:21:42.066621 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:42.066537    4943 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-vqzsp"
Oct 30 13:21:42.066684 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:42.066536    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-vqzsp" event={"ID":"6b9fc86d-7710-41b3-8d41-35cb717eb50e","Type":"ContainerDied","Data":"8db59a7efd7b42cde501a97b371d8429e86a1f57f77b62ed00959cb067a43748"}
Oct 30 13:21:42.066684 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:42.066670    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-vqzsp" event={"ID":"6b9fc86d-7710-41b3-8d41-35cb717eb50e","Type":"ContainerDied","Data":"c89f60127479615776b090e7e57f75c75e2f788753371b4cb5c00b0071497f64"}
Oct 30 13:21:42.068470 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:42.068402    4943 generic.go:356] "Generic (PLEG): container finished" podID="7bae0af9-fbeb-4480-9452-1abab86550e8" containerID="66b125e7d65ecd902d1fc405dc5610d87fe7ddca1c374f5c597feab53dd0dba7" exitCode=0
Oct 30 13:21:42.068470 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:42.068435    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-qzz29" event={"ID":"7bae0af9-fbeb-4480-9452-1abab86550e8","Type":"ContainerDied","Data":"66b125e7d65ecd902d1fc405dc5610d87fe7ddca1c374f5c597feab53dd0dba7"}
Oct 30 13:21:42.068470 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:42.068454    4943 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-qzz29"
Oct 30 13:21:42.068690 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:42.068485    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-qzz29" event={"ID":"7bae0af9-fbeb-4480-9452-1abab86550e8","Type":"ContainerDied","Data":"b318e29accd359fd5d6ffa4c003837d9647691d64c64b464f10c3711b6bfc94d"}
Oct 30 13:21:42.077751 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:42.077732    4943 scope.go:122] "RemoveContainer" containerID="ec6a6067cc5996f4434d40951dca64d038e1e6b22da9fd8cb27397b56344a9d9"
Oct 30 13:21:42.078107 ip-10-0-142-84 kubenswrapper[4943]: E1030 13:21:42.078043    4943 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ec6a6067cc5996f4434d40951dca64d038e1e6b22da9fd8cb27397b56344a9d9\": container with ID starting with ec6a6067cc5996f4434d40951dca64d038e1e6b22da9fd8cb27397b56344a9d9 not found: ID does not exist" containerID="ec6a6067cc5996f4434d40951dca64d038e1e6b22da9fd8cb27397b56344a9d9"
Oct 30 13:21:42.078107 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:42.078074    4943 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ec6a6067cc5996f4434d40951dca64d038e1e6b22da9fd8cb27397b56344a9d9"} err="failed to get container status \"ec6a6067cc5996f4434d40951dca64d038e1e6b22da9fd8cb27397b56344a9d9\": rpc error: code = NotFound desc = could not find container \"ec6a6067cc5996f4434d40951dca64d038e1e6b22da9fd8cb27397b56344a9d9\": container with ID starting with ec6a6067cc5996f4434d40951dca64d038e1e6b22da9fd8cb27397b56344a9d9 not found: ID does not exist"
Oct 30 13:21:42.078107 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:42.078091    4943 scope.go:122] "RemoveContainer" containerID="8db59a7efd7b42cde501a97b371d8429e86a1f57f77b62ed00959cb067a43748"
Oct 30 13:21:42.087674 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:42.087656    4943 scope.go:122] "RemoveContainer" containerID="8db59a7efd7b42cde501a97b371d8429e86a1f57f77b62ed00959cb067a43748"
Oct 30 13:21:42.087935 ip-10-0-142-84 kubenswrapper[4943]: E1030 13:21:42.087915    4943 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8db59a7efd7b42cde501a97b371d8429e86a1f57f77b62ed00959cb067a43748\": container with ID starting with 8db59a7efd7b42cde501a97b371d8429e86a1f57f77b62ed00959cb067a43748 not found: ID does not exist" containerID="8db59a7efd7b42cde501a97b371d8429e86a1f57f77b62ed00959cb067a43748"
Oct 30 13:21:42.088019 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:42.087946    4943 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8db59a7efd7b42cde501a97b371d8429e86a1f57f77b62ed00959cb067a43748"} err="failed to get container status \"8db59a7efd7b42cde501a97b371d8429e86a1f57f77b62ed00959cb067a43748\": rpc error: code = NotFound desc = could not find container \"8db59a7efd7b42cde501a97b371d8429e86a1f57f77b62ed00959cb067a43748\": container with ID starting with 8db59a7efd7b42cde501a97b371d8429e86a1f57f77b62ed00959cb067a43748 not found: ID does not exist"
Oct 30 13:21:42.088019 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:42.087969    4943 scope.go:122] "RemoveContainer" containerID="66b125e7d65ecd902d1fc405dc5610d87fe7ddca1c374f5c597feab53dd0dba7"
Oct 30 13:21:42.096812 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:42.096794    4943 scope.go:122] "RemoveContainer" containerID="c36000632379947fbb591c5b3f97b05877896b6ee819563caf1193313eaab073"
Oct 30 13:21:42.105717 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:42.105695    4943 scope.go:122] "RemoveContainer" containerID="66b125e7d65ecd902d1fc405dc5610d87fe7ddca1c374f5c597feab53dd0dba7"
Oct 30 13:21:42.105998 ip-10-0-142-84 kubenswrapper[4943]: E1030 13:21:42.105980    4943 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"66b125e7d65ecd902d1fc405dc5610d87fe7ddca1c374f5c597feab53dd0dba7\": container with ID starting with 66b125e7d65ecd902d1fc405dc5610d87fe7ddca1c374f5c597feab53dd0dba7 not found: ID does not exist" containerID="66b125e7d65ecd902d1fc405dc5610d87fe7ddca1c374f5c597feab53dd0dba7"
Oct 30 13:21:42.106050 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:42.106004    4943 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"66b125e7d65ecd902d1fc405dc5610d87fe7ddca1c374f5c597feab53dd0dba7"} err="failed to get container status \"66b125e7d65ecd902d1fc405dc5610d87fe7ddca1c374f5c597feab53dd0dba7\": rpc error: code = NotFound desc = could not find container \"66b125e7d65ecd902d1fc405dc5610d87fe7ddca1c374f5c597feab53dd0dba7\": container with ID starting with 66b125e7d65ecd902d1fc405dc5610d87fe7ddca1c374f5c597feab53dd0dba7 not found: ID does not exist"
Oct 30 13:21:42.106050 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:42.106020    4943 scope.go:122] "RemoveContainer" containerID="c36000632379947fbb591c5b3f97b05877896b6ee819563caf1193313eaab073"
Oct 30 13:21:42.106322 ip-10-0-142-84 kubenswrapper[4943]: E1030 13:21:42.106298    4943 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c36000632379947fbb591c5b3f97b05877896b6ee819563caf1193313eaab073\": container with ID starting with c36000632379947fbb591c5b3f97b05877896b6ee819563caf1193313eaab073 not found: ID does not exist" containerID="c36000632379947fbb591c5b3f97b05877896b6ee819563caf1193313eaab073"
Oct 30 13:21:42.106415 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:42.106329    4943 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c36000632379947fbb591c5b3f97b05877896b6ee819563caf1193313eaab073"} err="failed to get container status \"c36000632379947fbb591c5b3f97b05877896b6ee819563caf1193313eaab073\": rpc error: code = NotFound desc = could not find container \"c36000632379947fbb591c5b3f97b05877896b6ee819563caf1193313eaab073\": container with ID starting with c36000632379947fbb591c5b3f97b05877896b6ee819563caf1193313eaab073 not found: ID does not exist"
Oct 30 13:21:42.134926 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:42.134902    4943 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-vqzsp"]
Oct 30 13:21:42.150428 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:42.150402    4943 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-vqzsp"]
Oct 30 13:21:42.192529 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:42.192500    4943 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-586tt"]
Oct 30 13:21:42.196877 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:42.196848    4943 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-586tt"]
Oct 30 13:21:42.237042 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:42.237012    4943 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-qzz29"]
Oct 30 13:21:42.241238 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:42.241212    4943 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-qzz29"]
Oct 30 13:21:42.349495 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:42.349423    4943 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6b9fc86d-7710-41b3-8d41-35cb717eb50e" path="/var/lib/kubelet/pods/6b9fc86d-7710-41b3-8d41-35cb717eb50e/volumes"
Oct 30 13:21:42.349898 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:42.349884    4943 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7bae0af9-fbeb-4480-9452-1abab86550e8" path="/var/lib/kubelet/pods/7bae0af9-fbeb-4480-9452-1abab86550e8/volumes"
Oct 30 13:21:42.350345 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:42.350332    4943 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9c5b6db1-55db-447c-a3a9-2e24a9dc9dc6" path="/var/lib/kubelet/pods/9c5b6db1-55db-447c-a3a9-2e24a9dc9dc6/volumes"
Oct 30 13:21:43.805521 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:43.805481    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-hdmrm"]
Oct 30 13:21:46.678595 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:46.678511    4943 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w6-ab3176-2fdzj"]
Oct 30 13:21:46.692315 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:46.692283    4943 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w6-ab3176-2fdzj"]
Oct 30 13:21:47.996469 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:47.996427    4943 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-ab3176-5f84755579-6t85j" podUID="1d807dba-b7c2-412d-8be6-aa6a4f6db610" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 13:21:48.349138 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:48.349061    4943 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="87aca420-1995-4b47-a522-0d60b607db07" path="/var/lib/kubelet/pods/87aca420-1995-4b47-a522-0d60b607db07/volumes"
Oct 30 13:21:49.038393 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:49.038350    4943 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-67065e92-qcnbs"]
Oct 30 13:21:49.052090 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:49.052054    4943 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-67065e92-qcnbs"]
Oct 30 13:21:49.694298 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:49.694241    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-hdmrm"]
Oct 30 13:21:50.350429 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:50.350395    4943 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="3ab3da9a-ea4b-4638-996b-d1bf6a99a7d2" path="/var/lib/kubelet/pods/3ab3da9a-ea4b-4638-996b-d1bf6a99a7d2/volumes"
Oct 30 13:21:54.689706 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:54.689663    4943 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-l4l9h"]
Oct 30 13:21:54.690325 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:54.690021    4943 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-l4l9h" podUID="e9509589-96c1-4883-a552-86fd112c147f" containerName="istio-proxy" containerID="cri-o://9bd8d9a847930fe370a968a15e0e0a65b75b7489b60442efceddc0977539d8e3" gracePeriod=30
Oct 30 13:21:55.457338 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:55.457247    4943 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-ab3176-5f84755579-6t85j"
Oct 30 13:21:55.525159 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:55.525123    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1d807dba-b7c2-412d-8be6-aa6a4f6db610-kube-api-access-wwstb\" (UniqueName: \"kubernetes.io/projected/1d807dba-b7c2-412d-8be6-aa6a4f6db610-kube-api-access-wwstb\") pod \"1d807dba-b7c2-412d-8be6-aa6a4f6db610\" (UID: \"1d807dba-b7c2-412d-8be6-aa6a4f6db610\") "
Oct 30 13:21:55.525363 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:55.525219    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1d807dba-b7c2-412d-8be6-aa6a4f6db610-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1d807dba-b7c2-412d-8be6-aa6a4f6db610-plugins-config-volume\") pod \"1d807dba-b7c2-412d-8be6-aa6a4f6db610\" (UID: \"1d807dba-b7c2-412d-8be6-aa6a4f6db610\") "
Oct 30 13:21:55.525606 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:55.525581    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1d807dba-b7c2-412d-8be6-aa6a4f6db610-plugins-config-volume" pod "1d807dba-b7c2-412d-8be6-aa6a4f6db610" (UID: "1d807dba-b7c2-412d-8be6-aa6a4f6db610"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 13:21:55.527307 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:55.527277    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1d807dba-b7c2-412d-8be6-aa6a4f6db610-kube-api-access-wwstb" pod "1d807dba-b7c2-412d-8be6-aa6a4f6db610" (UID: "1d807dba-b7c2-412d-8be6-aa6a4f6db610"). InnerVolumeSpecName "kube-api-access-wwstb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:21:55.625889 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:55.625858    4943 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wwstb\" (UniqueName: \"kubernetes.io/projected/1d807dba-b7c2-412d-8be6-aa6a4f6db610-kube-api-access-wwstb\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:21:55.625889 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:55.625887    4943 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1d807dba-b7c2-412d-8be6-aa6a4f6db610-plugins-config-volume\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:21:56.173526 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:56.173489    4943 generic.go:356] "Generic (PLEG): container finished" podID="1d807dba-b7c2-412d-8be6-aa6a4f6db610" containerID="f9d2a89a5d0a684bffda22e5c8c2deb921f0b76ad3737c2dcc6ea495660b3321" exitCode=0
Oct 30 13:21:56.174009 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:56.173543    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-ab3176-5f84755579-6t85j" event={"ID":"1d807dba-b7c2-412d-8be6-aa6a4f6db610","Type":"ContainerDied","Data":"f9d2a89a5d0a684bffda22e5c8c2deb921f0b76ad3737c2dcc6ea495660b3321"}
Oct 30 13:21:56.174009 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:56.173560    4943 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-ab3176-5f84755579-6t85j"
Oct 30 13:21:56.174009 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:56.173633    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-ab3176-5f84755579-6t85j" event={"ID":"1d807dba-b7c2-412d-8be6-aa6a4f6db610","Type":"ContainerDied","Data":"2bf03f26f8cd94649bd28b9422993ebcc32a121da79aa4b78a2395621e7dc175"}
Oct 30 13:21:56.174009 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:56.173669    4943 scope.go:122] "RemoveContainer" containerID="f9d2a89a5d0a684bffda22e5c8c2deb921f0b76ad3737c2dcc6ea495660b3321"
Oct 30 13:21:56.184622 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:56.184601    4943 scope.go:122] "RemoveContainer" containerID="f9d2a89a5d0a684bffda22e5c8c2deb921f0b76ad3737c2dcc6ea495660b3321"
Oct 30 13:21:56.184913 ip-10-0-142-84 kubenswrapper[4943]: E1030 13:21:56.184889    4943 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f9d2a89a5d0a684bffda22e5c8c2deb921f0b76ad3737c2dcc6ea495660b3321\": container with ID starting with f9d2a89a5d0a684bffda22e5c8c2deb921f0b76ad3737c2dcc6ea495660b3321 not found: ID does not exist" containerID="f9d2a89a5d0a684bffda22e5c8c2deb921f0b76ad3737c2dcc6ea495660b3321"
Oct 30 13:21:56.184960 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:56.184921    4943 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f9d2a89a5d0a684bffda22e5c8c2deb921f0b76ad3737c2dcc6ea495660b3321"} err="failed to get container status \"f9d2a89a5d0a684bffda22e5c8c2deb921f0b76ad3737c2dcc6ea495660b3321\": rpc error: code = NotFound desc = could not find container \"f9d2a89a5d0a684bffda22e5c8c2deb921f0b76ad3737c2dcc6ea495660b3321\": container with ID starting with f9d2a89a5d0a684bffda22e5c8c2deb921f0b76ad3737c2dcc6ea495660b3321 not found: ID does not exist"
Oct 30 13:21:56.232797 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:56.232765    4943 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w6-ab3176-5f84755579-6t85j"]
Oct 30 13:21:56.246136 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:56.246107    4943 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w6-ab3176-5f84755579-6t85j"]
Oct 30 13:21:56.349703 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:21:56.349675    4943 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1d807dba-b7c2-412d-8be6-aa6a4f6db610" path="/var/lib/kubelet/pods/1d807dba-b7c2-412d-8be6-aa6a4f6db610/volumes"
Oct 30 13:22:00.033989 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:00.033962    4943 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-l4l9h"
Oct 30 13:22:00.073153 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:00.072119    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e9509589-96c1-4883-a552-86fd112c147f-kube-api-access-2c9j4\" (UniqueName: \"kubernetes.io/projected/e9509589-96c1-4883-a552-86fd112c147f-kube-api-access-2c9j4\") pod \"e9509589-96c1-4883-a552-86fd112c147f\" (UID: \"e9509589-96c1-4883-a552-86fd112c147f\") "
Oct 30 13:22:00.073153 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:00.072181    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e9509589-96c1-4883-a552-86fd112c147f-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e9509589-96c1-4883-a552-86fd112c147f-istio-envoy\") pod \"e9509589-96c1-4883-a552-86fd112c147f\" (UID: \"e9509589-96c1-4883-a552-86fd112c147f\") "
Oct 30 13:22:00.073153 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:00.072272    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e9509589-96c1-4883-a552-86fd112c147f-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e9509589-96c1-4883-a552-86fd112c147f-istio-data\") pod \"e9509589-96c1-4883-a552-86fd112c147f\" (UID: \"e9509589-96c1-4883-a552-86fd112c147f\") "
Oct 30 13:22:00.073153 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:00.072306    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e9509589-96c1-4883-a552-86fd112c147f-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e9509589-96c1-4883-a552-86fd112c147f-credential-socket\") pod \"e9509589-96c1-4883-a552-86fd112c147f\" (UID: \"e9509589-96c1-4883-a552-86fd112c147f\") "
Oct 30 13:22:00.073153 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:00.072347    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e9509589-96c1-4883-a552-86fd112c147f-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e9509589-96c1-4883-a552-86fd112c147f-istiod-ca-cert\") pod \"e9509589-96c1-4883-a552-86fd112c147f\" (UID: \"e9509589-96c1-4883-a552-86fd112c147f\") "
Oct 30 13:22:00.073153 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:00.072410    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/e9509589-96c1-4883-a552-86fd112c147f-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e9509589-96c1-4883-a552-86fd112c147f-istio-podinfo\") pod \"e9509589-96c1-4883-a552-86fd112c147f\" (UID: \"e9509589-96c1-4883-a552-86fd112c147f\") "
Oct 30 13:22:00.073153 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:00.072440    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e9509589-96c1-4883-a552-86fd112c147f-istio-token\" (UniqueName: \"kubernetes.io/projected/e9509589-96c1-4883-a552-86fd112c147f-istio-token\") pod \"e9509589-96c1-4883-a552-86fd112c147f\" (UID: \"e9509589-96c1-4883-a552-86fd112c147f\") "
Oct 30 13:22:00.073153 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:00.072490    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e9509589-96c1-4883-a552-86fd112c147f-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e9509589-96c1-4883-a552-86fd112c147f-workload-socket\") pod \"e9509589-96c1-4883-a552-86fd112c147f\" (UID: \"e9509589-96c1-4883-a552-86fd112c147f\") "
Oct 30 13:22:00.073153 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:00.072539    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e9509589-96c1-4883-a552-86fd112c147f-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e9509589-96c1-4883-a552-86fd112c147f-workload-certs\") pod \"e9509589-96c1-4883-a552-86fd112c147f\" (UID: \"e9509589-96c1-4883-a552-86fd112c147f\") "
Oct 30 13:22:00.074081 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:00.073848    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e9509589-96c1-4883-a552-86fd112c147f-workload-certs" pod "e9509589-96c1-4883-a552-86fd112c147f" (UID: "e9509589-96c1-4883-a552-86fd112c147f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:22:00.074646 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:00.074377    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e9509589-96c1-4883-a552-86fd112c147f-istiod-ca-cert" pod "e9509589-96c1-4883-a552-86fd112c147f" (UID: "e9509589-96c1-4883-a552-86fd112c147f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 13:22:00.075591 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:00.075502    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e9509589-96c1-4883-a552-86fd112c147f-istio-data" pod "e9509589-96c1-4883-a552-86fd112c147f" (UID: "e9509589-96c1-4883-a552-86fd112c147f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:22:00.075591 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:00.075565    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e9509589-96c1-4883-a552-86fd112c147f-credential-socket" pod "e9509589-96c1-4883-a552-86fd112c147f" (UID: "e9509589-96c1-4883-a552-86fd112c147f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:22:00.075978 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:00.075957    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e9509589-96c1-4883-a552-86fd112c147f-workload-socket" pod "e9509589-96c1-4883-a552-86fd112c147f" (UID: "e9509589-96c1-4883-a552-86fd112c147f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:22:00.097782 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:00.097583    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e9509589-96c1-4883-a552-86fd112c147f-istio-envoy" pod "e9509589-96c1-4883-a552-86fd112c147f" (UID: "e9509589-96c1-4883-a552-86fd112c147f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:22:00.120440 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:00.120389    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e9509589-96c1-4883-a552-86fd112c147f-istio-token" pod "e9509589-96c1-4883-a552-86fd112c147f" (UID: "e9509589-96c1-4883-a552-86fd112c147f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:22:00.121408 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:00.121365    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e9509589-96c1-4883-a552-86fd112c147f-istio-podinfo" pod "e9509589-96c1-4883-a552-86fd112c147f" (UID: "e9509589-96c1-4883-a552-86fd112c147f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 13:22:00.121687 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:00.121644    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e9509589-96c1-4883-a552-86fd112c147f-kube-api-access-2c9j4" pod "e9509589-96c1-4883-a552-86fd112c147f" (UID: "e9509589-96c1-4883-a552-86fd112c147f"). InnerVolumeSpecName "kube-api-access-2c9j4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:22:00.174015 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:00.173973    4943 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2c9j4\" (UniqueName: \"kubernetes.io/projected/e9509589-96c1-4883-a552-86fd112c147f-kube-api-access-2c9j4\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:22:00.174015 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:00.174020    4943 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e9509589-96c1-4883-a552-86fd112c147f-istio-envoy\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:22:00.174299 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:00.174039    4943 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e9509589-96c1-4883-a552-86fd112c147f-istio-data\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:22:00.174299 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:00.174054    4943 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e9509589-96c1-4883-a552-86fd112c147f-credential-socket\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:22:00.174299 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:00.174070    4943 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e9509589-96c1-4883-a552-86fd112c147f-istiod-ca-cert\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:22:00.174299 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:00.174084    4943 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e9509589-96c1-4883-a552-86fd112c147f-istio-podinfo\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:22:00.174299 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:00.174099    4943 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e9509589-96c1-4883-a552-86fd112c147f-istio-token\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:22:00.174299 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:00.174112    4943 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e9509589-96c1-4883-a552-86fd112c147f-workload-socket\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:22:00.174299 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:00.174125    4943 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e9509589-96c1-4883-a552-86fd112c147f-workload-certs\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:22:00.231120 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:00.231086    4943 generic.go:356] "Generic (PLEG): container finished" podID="e9509589-96c1-4883-a552-86fd112c147f" containerID="9bd8d9a847930fe370a968a15e0e0a65b75b7489b60442efceddc0977539d8e3" exitCode=0
Oct 30 13:22:00.231327 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:00.231148    4943 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-l4l9h"
Oct 30 13:22:00.231327 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:00.231165    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-l4l9h" event={"ID":"e9509589-96c1-4883-a552-86fd112c147f","Type":"ContainerDied","Data":"9bd8d9a847930fe370a968a15e0e0a65b75b7489b60442efceddc0977539d8e3"}
Oct 30 13:22:00.231327 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:00.231220    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-l4l9h" event={"ID":"e9509589-96c1-4883-a552-86fd112c147f","Type":"ContainerDied","Data":"5ca1a5fa65a9e91ded7acf02a70eca156a7d250dc3286bc3ac05ca5d5130c262"}
Oct 30 13:22:00.231327 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:00.231243    4943 scope.go:122] "RemoveContainer" containerID="9bd8d9a847930fe370a968a15e0e0a65b75b7489b60442efceddc0977539d8e3"
Oct 30 13:22:00.242440 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:00.242423    4943 scope.go:122] "RemoveContainer" containerID="8ba842ec132925a085d6d2ecbee87fb111bd4c7e6c15faada2102de24f212d34"
Oct 30 13:22:00.252202 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:00.252179    4943 scope.go:122] "RemoveContainer" containerID="9bd8d9a847930fe370a968a15e0e0a65b75b7489b60442efceddc0977539d8e3"
Oct 30 13:22:00.252502 ip-10-0-142-84 kubenswrapper[4943]: E1030 13:22:00.252481    4943 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9bd8d9a847930fe370a968a15e0e0a65b75b7489b60442efceddc0977539d8e3\": container with ID starting with 9bd8d9a847930fe370a968a15e0e0a65b75b7489b60442efceddc0977539d8e3 not found: ID does not exist" containerID="9bd8d9a847930fe370a968a15e0e0a65b75b7489b60442efceddc0977539d8e3"
Oct 30 13:22:00.252570 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:00.252512    4943 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9bd8d9a847930fe370a968a15e0e0a65b75b7489b60442efceddc0977539d8e3"} err="failed to get container status \"9bd8d9a847930fe370a968a15e0e0a65b75b7489b60442efceddc0977539d8e3\": rpc error: code = NotFound desc = could not find container \"9bd8d9a847930fe370a968a15e0e0a65b75b7489b60442efceddc0977539d8e3\": container with ID starting with 9bd8d9a847930fe370a968a15e0e0a65b75b7489b60442efceddc0977539d8e3 not found: ID does not exist"
Oct 30 13:22:00.252570 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:00.252529    4943 scope.go:122] "RemoveContainer" containerID="8ba842ec132925a085d6d2ecbee87fb111bd4c7e6c15faada2102de24f212d34"
Oct 30 13:22:00.252774 ip-10-0-142-84 kubenswrapper[4943]: E1030 13:22:00.252753    4943 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8ba842ec132925a085d6d2ecbee87fb111bd4c7e6c15faada2102de24f212d34\": container with ID starting with 8ba842ec132925a085d6d2ecbee87fb111bd4c7e6c15faada2102de24f212d34 not found: ID does not exist" containerID="8ba842ec132925a085d6d2ecbee87fb111bd4c7e6c15faada2102de24f212d34"
Oct 30 13:22:00.252835 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:00.252784    4943 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8ba842ec132925a085d6d2ecbee87fb111bd4c7e6c15faada2102de24f212d34"} err="failed to get container status \"8ba842ec132925a085d6d2ecbee87fb111bd4c7e6c15faada2102de24f212d34\": rpc error: code = NotFound desc = could not find container \"8ba842ec132925a085d6d2ecbee87fb111bd4c7e6c15faada2102de24f212d34\": container with ID starting with 8ba842ec132925a085d6d2ecbee87fb111bd4c7e6c15faada2102de24f212d34 not found: ID does not exist"
Oct 30 13:22:00.302513 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:00.302472    4943 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-l4l9h"]
Oct 30 13:22:00.307491 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:00.307458    4943 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-l4l9h"]
Oct 30 13:22:00.349353 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:00.349320    4943 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e9509589-96c1-4883-a552-86fd112c147f" path="/var/lib/kubelet/pods/e9509589-96c1-4883-a552-86fd112c147f/volumes"
Oct 30 13:22:00.974466 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:00.974414    4943 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-c9ffffc7f-q58qm container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 13:22:00.974660 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:00.974527    4943 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-q58qm" podUID="883f1e32-141f-45e4-9f0e-31f9231fab0d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:22:01.170500 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:01.170408    4943 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-c9ffffc7f-xs2vr container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.49:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 13:22:01.170997 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:01.170502    4943 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-xs2vr" podUID="e3af0d52-346e-48d7-ac7d-8f2aeee62448" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.49:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:22:04.533773 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:04.533746    4943 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ajkfrv_17f5a8d2-67e0-4489-8734-00f0d4483788/extract/0.log"
Oct 30 13:22:04.543926 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:04.543904    4943 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ajkfrv_17f5a8d2-67e0-4489-8734-00f0d4483788/extract/0.log"
Oct 30 13:22:04.544289 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:04.544245    4943 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-samples-operator_cluster-samples-operator-6ddb6c8b94-7dtcd_23297e74-48f9-47b7-bb5a-80189bdbf2c4/cluster-samples-operator/0.log"
Oct 30 13:22:04.546605 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:04.546581    4943 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-8fqfr_d8930ee7-0ae0-48b5-8ce3-cb8820355cef/console-operator/2.log"
Oct 30 13:22:04.553567 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:04.553549    4943 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-samples-operator_cluster-samples-operator-6ddb6c8b94-7dtcd_23297e74-48f9-47b7-bb5a-80189bdbf2c4/cluster-samples-operator/0.log"
Oct 30 13:22:04.555716 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:04.555696    4943 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-8fqfr_d8930ee7-0ae0-48b5-8ce3-cb8820355cef/console-operator/2.log"
Oct 30 13:22:05.955103 ip-10-0-142-84 kubenswrapper[4943]: E1030 13:22:05.955068    4943 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-knqz6" containerName="extractor"
Oct 30 13:22:07.466567 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:07.466534    4943 scope.go:122] "RemoveContainer" containerID="e9709984d4dc46fbb9bea79b5a24418bbb2a51b79d5a15850579e8994bf4c126"
Oct 30 13:22:07.480575 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:07.480552    4943 scope.go:122] "RemoveContainer" containerID="a637a84f7281a0834c1f26089f096423f32715fb8969c01ee7d193735716e06a"
Oct 30 13:22:07.489376 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:07.489357    4943 scope.go:122] "RemoveContainer" containerID="59aae9c206b0c61474afc19f14c1cb183b0001a2e0defd361fcf50552e93c494"
Oct 30 13:22:07.498277 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:07.498237    4943 scope.go:122] "RemoveContainer" containerID="03b70ab6c2da08562cd652fbdaf1ead3503d608089d8bb67dcac1a69606a9588"
Oct 30 13:22:07.506970 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:07.506941    4943 scope.go:122] "RemoveContainer" containerID="223db64525dfc66384575504caac98120a59bdb77baa254d263374796ab94914"
Oct 30 13:22:07.515518 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:07.515503    4943 scope.go:122] "RemoveContainer" containerID="a8094c9252e39be8e23abff211fa0dc08678a24fefd1f5bb86af5ae8e43f765a"
Oct 30 13:22:21.057766 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:21.057727    4943 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-pd5gs"]
Oct 30 13:22:21.058163 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:21.058149    4943 cpu_manager.go:414] "containerMap: removing container" podUID="da306792-4f80-4df4-aa44-82991ab856e4" containerName="main"
Oct 30 13:22:21.058211 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:21.058164    4943 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="da306792-4f80-4df4-aa44-82991ab856e4" containerName="main"
Oct 30 13:22:21.058211 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:21.058179    4943 cpu_manager.go:414] "containerMap: removing container" podUID="7bae0af9-fbeb-4480-9452-1abab86550e8" containerName="istio-proxy"
Oct 30 13:22:21.058211 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:21.058184    4943 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7bae0af9-fbeb-4480-9452-1abab86550e8" containerName="istio-proxy"
Oct 30 13:22:21.058211 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:21.058192    4943 cpu_manager.go:414] "containerMap: removing container" podUID="92a51875-955c-40bd-994e-9ab0e8f46938" containerName="revoke-keys"
Oct 30 13:22:21.058211 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:21.058197    4943 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="92a51875-955c-40bd-994e-9ab0e8f46938" containerName="revoke-keys"
Oct 30 13:22:21.058211 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:21.058207    4943 cpu_manager.go:414] "containerMap: removing container" podUID="e9509589-96c1-4883-a552-86fd112c147f" containerName="istio-proxy"
Oct 30 13:22:21.058211 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:21.058212    4943 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e9509589-96c1-4883-a552-86fd112c147f" containerName="istio-proxy"
Oct 30 13:22:21.058459 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:21.058218    4943 cpu_manager.go:414] "containerMap: removing container" podUID="1d807dba-b7c2-412d-8be6-aa6a4f6db610" containerName="payload-pre-processing"
Oct 30 13:22:21.058459 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:21.058223    4943 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1d807dba-b7c2-412d-8be6-aa6a4f6db610" containerName="payload-pre-processing"
Oct 30 13:22:21.058459 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:21.058229    4943 cpu_manager.go:414] "containerMap: removing container" podUID="6b9fc86d-7710-41b3-8d41-35cb717eb50e" containerName="istio-proxy"
Oct 30 13:22:21.058459 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:21.058234    4943 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6b9fc86d-7710-41b3-8d41-35cb717eb50e" containerName="istio-proxy"
Oct 30 13:22:21.058459 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:21.058276    4943 cpu_manager.go:414] "containerMap: removing container" podUID="9c5b6db1-55db-447c-a3a9-2e24a9dc9dc6" containerName="istio-proxy"
Oct 30 13:22:21.058459 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:21.058283    4943 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9c5b6db1-55db-447c-a3a9-2e24a9dc9dc6" containerName="istio-proxy"
Oct 30 13:22:21.058459 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:21.058366    4943 memory_manager.go:365] "RemoveStaleState removing state" podUID="7bae0af9-fbeb-4480-9452-1abab86550e8" containerName="istio-proxy"
Oct 30 13:22:21.058459 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:21.058375    4943 memory_manager.go:365] "RemoveStaleState removing state" podUID="92a51875-955c-40bd-994e-9ab0e8f46938" containerName="revoke-keys"
Oct 30 13:22:21.058459 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:21.058381    4943 memory_manager.go:365] "RemoveStaleState removing state" podUID="e9509589-96c1-4883-a552-86fd112c147f" containerName="istio-proxy"
Oct 30 13:22:21.058459 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:21.058386    4943 memory_manager.go:365] "RemoveStaleState removing state" podUID="1d807dba-b7c2-412d-8be6-aa6a4f6db610" containerName="payload-pre-processing"
Oct 30 13:22:21.058459 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:21.058392    4943 memory_manager.go:365] "RemoveStaleState removing state" podUID="6b9fc86d-7710-41b3-8d41-35cb717eb50e" containerName="istio-proxy"
Oct 30 13:22:21.058459 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:21.058401    4943 memory_manager.go:365] "RemoveStaleState removing state" podUID="7bae0af9-fbeb-4480-9452-1abab86550e8" containerName="istio-proxy"
Oct 30 13:22:21.058459 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:21.058409    4943 memory_manager.go:365] "RemoveStaleState removing state" podUID="e9509589-96c1-4883-a552-86fd112c147f" containerName="istio-proxy"
Oct 30 13:22:21.058459 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:21.058416    4943 memory_manager.go:365] "RemoveStaleState removing state" podUID="9c5b6db1-55db-447c-a3a9-2e24a9dc9dc6" containerName="istio-proxy"
Oct 30 13:22:21.058459 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:21.058423    4943 memory_manager.go:365] "RemoveStaleState removing state" podUID="da306792-4f80-4df4-aa44-82991ab856e4" containerName="main"
Oct 30 13:22:21.064505 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:21.064485    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-pd5gs"
Oct 30 13:22:21.087289 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:21.083983    4943 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-6h9t4"]
Oct 30 13:22:21.087289 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:21.084609    4943 cpu_manager.go:414] "containerMap: removing container" podUID="7bae0af9-fbeb-4480-9452-1abab86550e8" containerName="istio-proxy"
Oct 30 13:22:21.087289 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:21.084627    4943 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7bae0af9-fbeb-4480-9452-1abab86550e8" containerName="istio-proxy"
Oct 30 13:22:21.087289 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:21.084649    4943 cpu_manager.go:414] "containerMap: removing container" podUID="e9509589-96c1-4883-a552-86fd112c147f" containerName="istio-proxy"
Oct 30 13:22:21.087289 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:21.084657    4943 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e9509589-96c1-4883-a552-86fd112c147f" containerName="istio-proxy"
Oct 30 13:22:21.088193 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:21.088161    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-6h9t4"
Oct 30 13:22:21.098245 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:21.098216    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-pd5gs"]
Oct 30 13:22:21.116975 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:21.116948    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-6h9t4"]
Oct 30 13:22:21.168534 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:21.168503    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/28d7688e-c812-4fcc-b223-151368afe767-istio-token\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-pd5gs\" (UID: \"28d7688e-c812-4fcc-b223-151368afe767\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-pd5gs"
Oct 30 13:22:21.168534 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:21.168535    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/acf3c9ea-1f3e-40d5-88cd-2b76d8897517-istio-envoy\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-6h9t4\" (UID: \"acf3c9ea-1f3e-40d5-88cd-2b76d8897517\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-6h9t4"
Oct 30 13:22:21.168709 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:21.168557    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bqk9t\" (UniqueName: \"kubernetes.io/projected/28d7688e-c812-4fcc-b223-151368afe767-kube-api-access-bqk9t\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-pd5gs\" (UID: \"28d7688e-c812-4fcc-b223-151368afe767\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-pd5gs"
Oct 30 13:22:21.168709 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:21.168658    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/28d7688e-c812-4fcc-b223-151368afe767-istio-data\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-pd5gs\" (UID: \"28d7688e-c812-4fcc-b223-151368afe767\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-pd5gs"
Oct 30 13:22:21.168709 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:21.168681    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/28d7688e-c812-4fcc-b223-151368afe767-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-pd5gs\" (UID: \"28d7688e-c812-4fcc-b223-151368afe767\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-pd5gs"
Oct 30 13:22:21.168709 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:21.168699    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/acf3c9ea-1f3e-40d5-88cd-2b76d8897517-istio-podinfo\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-6h9t4\" (UID: \"acf3c9ea-1f3e-40d5-88cd-2b76d8897517\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-6h9t4"
Oct 30 13:22:21.168841 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:21.168730    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/28d7688e-c812-4fcc-b223-151368afe767-credential-socket\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-pd5gs\" (UID: \"28d7688e-c812-4fcc-b223-151368afe767\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-pd5gs"
Oct 30 13:22:21.168841 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:21.168809    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/acf3c9ea-1f3e-40d5-88cd-2b76d8897517-istio-token\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-6h9t4\" (UID: \"acf3c9ea-1f3e-40d5-88cd-2b76d8897517\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-6h9t4"
Oct 30 13:22:21.168914 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:21.168887    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/28d7688e-c812-4fcc-b223-151368afe767-istio-podinfo\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-pd5gs\" (UID: \"28d7688e-c812-4fcc-b223-151368afe767\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-pd5gs"
Oct 30 13:22:21.168949 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:21.168913    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/acf3c9ea-1f3e-40d5-88cd-2b76d8897517-istio-data\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-6h9t4\" (UID: \"acf3c9ea-1f3e-40d5-88cd-2b76d8897517\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-6h9t4"
Oct 30 13:22:21.168949 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:21.168936    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/28d7688e-c812-4fcc-b223-151368afe767-workload-certs\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-pd5gs\" (UID: \"28d7688e-c812-4fcc-b223-151368afe767\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-pd5gs"
Oct 30 13:22:21.169011 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:21.168952    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/acf3c9ea-1f3e-40d5-88cd-2b76d8897517-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-6h9t4\" (UID: \"acf3c9ea-1f3e-40d5-88cd-2b76d8897517\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-6h9t4"
Oct 30 13:22:21.169066 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:21.169051    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/acf3c9ea-1f3e-40d5-88cd-2b76d8897517-workload-certs\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-6h9t4\" (UID: \"acf3c9ea-1f3e-40d5-88cd-2b76d8897517\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-6h9t4"
Oct 30 13:22:21.169103 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:21.169078    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/28d7688e-c812-4fcc-b223-151368afe767-workload-socket\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-pd5gs\" (UID: \"28d7688e-c812-4fcc-b223-151368afe767\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-pd5gs"
Oct 30 13:22:21.169103 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:21.169094    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rhwcd\" (UniqueName: \"kubernetes.io/projected/acf3c9ea-1f3e-40d5-88cd-2b76d8897517-kube-api-access-rhwcd\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-6h9t4\" (UID: \"acf3c9ea-1f3e-40d5-88cd-2b76d8897517\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-6h9t4"
Oct 30 13:22:21.169166 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:21.169121    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/28d7688e-c812-4fcc-b223-151368afe767-istio-envoy\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-pd5gs\" (UID: \"28d7688e-c812-4fcc-b223-151368afe767\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-pd5gs"
Oct 30 13:22:21.169197 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:21.169183    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/acf3c9ea-1f3e-40d5-88cd-2b76d8897517-credential-socket\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-6h9t4\" (UID: \"acf3c9ea-1f3e-40d5-88cd-2b76d8897517\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-6h9t4"
Oct 30 13:22:21.169233 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:21.169212    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/acf3c9ea-1f3e-40d5-88cd-2b76d8897517-workload-socket\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-6h9t4\" (UID: \"acf3c9ea-1f3e-40d5-88cd-2b76d8897517\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-6h9t4"
Oct 30 13:22:21.269687 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:21.269651    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/acf3c9ea-1f3e-40d5-88cd-2b76d8897517-workload-certs\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-6h9t4\" (UID: \"acf3c9ea-1f3e-40d5-88cd-2b76d8897517\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-6h9t4"
Oct 30 13:22:21.269864 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:21.269694    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/28d7688e-c812-4fcc-b223-151368afe767-workload-socket\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-pd5gs\" (UID: \"28d7688e-c812-4fcc-b223-151368afe767\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-pd5gs"
Oct 30 13:22:21.269864 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:21.269721    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rhwcd\" (UniqueName: \"kubernetes.io/projected/acf3c9ea-1f3e-40d5-88cd-2b76d8897517-kube-api-access-rhwcd\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-6h9t4\" (UID: \"acf3c9ea-1f3e-40d5-88cd-2b76d8897517\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-6h9t4"
Oct 30 13:22:21.269864 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:21.269756    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/28d7688e-c812-4fcc-b223-151368afe767-istio-envoy\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-pd5gs\" (UID: \"28d7688e-c812-4fcc-b223-151368afe767\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-pd5gs"
Oct 30 13:22:21.269864 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:21.269785    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/acf3c9ea-1f3e-40d5-88cd-2b76d8897517-credential-socket\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-6h9t4\" (UID: \"acf3c9ea-1f3e-40d5-88cd-2b76d8897517\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-6h9t4"
Oct 30 13:22:21.269864 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:21.269840    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/acf3c9ea-1f3e-40d5-88cd-2b76d8897517-workload-socket\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-6h9t4\" (UID: \"acf3c9ea-1f3e-40d5-88cd-2b76d8897517\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-6h9t4"
Oct 30 13:22:21.270135 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:21.269898    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/28d7688e-c812-4fcc-b223-151368afe767-istio-token\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-pd5gs\" (UID: \"28d7688e-c812-4fcc-b223-151368afe767\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-pd5gs"
Oct 30 13:22:21.270135 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:21.269927    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/acf3c9ea-1f3e-40d5-88cd-2b76d8897517-istio-envoy\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-6h9t4\" (UID: \"acf3c9ea-1f3e-40d5-88cd-2b76d8897517\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-6h9t4"
Oct 30 13:22:21.270135 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:21.269954    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bqk9t\" (UniqueName: \"kubernetes.io/projected/28d7688e-c812-4fcc-b223-151368afe767-kube-api-access-bqk9t\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-pd5gs\" (UID: \"28d7688e-c812-4fcc-b223-151368afe767\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-pd5gs"
Oct 30 13:22:21.270135 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:21.270003    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/28d7688e-c812-4fcc-b223-151368afe767-istio-data\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-pd5gs\" (UID: \"28d7688e-c812-4fcc-b223-151368afe767\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-pd5gs"
Oct 30 13:22:21.270135 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:21.270026    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/28d7688e-c812-4fcc-b223-151368afe767-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-pd5gs\" (UID: \"28d7688e-c812-4fcc-b223-151368afe767\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-pd5gs"
Oct 30 13:22:21.270135 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:21.270058    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/acf3c9ea-1f3e-40d5-88cd-2b76d8897517-istio-podinfo\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-6h9t4\" (UID: \"acf3c9ea-1f3e-40d5-88cd-2b76d8897517\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-6h9t4"
Oct 30 13:22:21.270502 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:21.270143    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/acf3c9ea-1f3e-40d5-88cd-2b76d8897517-workload-certs\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-6h9t4\" (UID: \"acf3c9ea-1f3e-40d5-88cd-2b76d8897517\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-6h9t4"
Oct 30 13:22:21.270502 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:21.270164    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/acf3c9ea-1f3e-40d5-88cd-2b76d8897517-credential-socket\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-6h9t4\" (UID: \"acf3c9ea-1f3e-40d5-88cd-2b76d8897517\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-6h9t4"
Oct 30 13:22:21.270502 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:21.270145    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/28d7688e-c812-4fcc-b223-151368afe767-workload-socket\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-pd5gs\" (UID: \"28d7688e-c812-4fcc-b223-151368afe767\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-pd5gs"
Oct 30 13:22:21.270502 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:21.270189    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/28d7688e-c812-4fcc-b223-151368afe767-credential-socket\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-pd5gs\" (UID: \"28d7688e-c812-4fcc-b223-151368afe767\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-pd5gs"
Oct 30 13:22:21.270502 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:21.270236    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/acf3c9ea-1f3e-40d5-88cd-2b76d8897517-istio-token\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-6h9t4\" (UID: \"acf3c9ea-1f3e-40d5-88cd-2b76d8897517\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-6h9t4"
Oct 30 13:22:21.270502 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:21.270234    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/acf3c9ea-1f3e-40d5-88cd-2b76d8897517-workload-socket\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-6h9t4\" (UID: \"acf3c9ea-1f3e-40d5-88cd-2b76d8897517\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-6h9t4"
Oct 30 13:22:21.270502 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:21.270307    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/28d7688e-c812-4fcc-b223-151368afe767-istio-podinfo\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-pd5gs\" (UID: \"28d7688e-c812-4fcc-b223-151368afe767\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-pd5gs"
Oct 30 13:22:21.270502 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:21.270346    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/acf3c9ea-1f3e-40d5-88cd-2b76d8897517-istio-data\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-6h9t4\" (UID: \"acf3c9ea-1f3e-40d5-88cd-2b76d8897517\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-6h9t4"
Oct 30 13:22:21.270502 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:21.270394    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/28d7688e-c812-4fcc-b223-151368afe767-workload-certs\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-pd5gs\" (UID: \"28d7688e-c812-4fcc-b223-151368afe767\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-pd5gs"
Oct 30 13:22:21.270502 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:21.270419    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/acf3c9ea-1f3e-40d5-88cd-2b76d8897517-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-6h9t4\" (UID: \"acf3c9ea-1f3e-40d5-88cd-2b76d8897517\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-6h9t4"
Oct 30 13:22:21.271041 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:21.270737    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/acf3c9ea-1f3e-40d5-88cd-2b76d8897517-istio-data\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-6h9t4\" (UID: \"acf3c9ea-1f3e-40d5-88cd-2b76d8897517\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-6h9t4"
Oct 30 13:22:21.271041 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:21.270811    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/28d7688e-c812-4fcc-b223-151368afe767-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-pd5gs\" (UID: \"28d7688e-c812-4fcc-b223-151368afe767\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-pd5gs"
Oct 30 13:22:21.271191 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:21.271166    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/acf3c9ea-1f3e-40d5-88cd-2b76d8897517-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-6h9t4\" (UID: \"acf3c9ea-1f3e-40d5-88cd-2b76d8897517\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-6h9t4"
Oct 30 13:22:21.271291 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:21.271244    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/28d7688e-c812-4fcc-b223-151368afe767-workload-certs\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-pd5gs\" (UID: \"28d7688e-c812-4fcc-b223-151368afe767\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-pd5gs"
Oct 30 13:22:21.271357 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:21.271296    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/28d7688e-c812-4fcc-b223-151368afe767-credential-socket\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-pd5gs\" (UID: \"28d7688e-c812-4fcc-b223-151368afe767\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-pd5gs"
Oct 30 13:22:21.271357 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:21.271344    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/28d7688e-c812-4fcc-b223-151368afe767-istio-data\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-pd5gs\" (UID: \"28d7688e-c812-4fcc-b223-151368afe767\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-pd5gs"
Oct 30 13:22:21.272938 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:21.272911    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/acf3c9ea-1f3e-40d5-88cd-2b76d8897517-istio-envoy\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-6h9t4\" (UID: \"acf3c9ea-1f3e-40d5-88cd-2b76d8897517\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-6h9t4"
Oct 30 13:22:21.273148 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:21.273127    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/acf3c9ea-1f3e-40d5-88cd-2b76d8897517-istio-podinfo\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-6h9t4\" (UID: \"acf3c9ea-1f3e-40d5-88cd-2b76d8897517\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-6h9t4"
Oct 30 13:22:21.273219 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:21.273167    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/28d7688e-c812-4fcc-b223-151368afe767-istio-podinfo\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-pd5gs\" (UID: \"28d7688e-c812-4fcc-b223-151368afe767\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-pd5gs"
Oct 30 13:22:21.273420 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:21.273405    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/28d7688e-c812-4fcc-b223-151368afe767-istio-envoy\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-pd5gs\" (UID: \"28d7688e-c812-4fcc-b223-151368afe767\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-pd5gs"
Oct 30 13:22:21.304794 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:21.304763    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/acf3c9ea-1f3e-40d5-88cd-2b76d8897517-istio-token\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-6h9t4\" (UID: \"acf3c9ea-1f3e-40d5-88cd-2b76d8897517\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-6h9t4"
Oct 30 13:22:21.304929 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:21.304799    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rhwcd\" (UniqueName: \"kubernetes.io/projected/acf3c9ea-1f3e-40d5-88cd-2b76d8897517-kube-api-access-rhwcd\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-6h9t4\" (UID: \"acf3c9ea-1f3e-40d5-88cd-2b76d8897517\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-6h9t4"
Oct 30 13:22:21.304929 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:21.304903    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bqk9t\" (UniqueName: \"kubernetes.io/projected/28d7688e-c812-4fcc-b223-151368afe767-kube-api-access-bqk9t\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-pd5gs\" (UID: \"28d7688e-c812-4fcc-b223-151368afe767\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-pd5gs"
Oct 30 13:22:21.317105 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:21.317051    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/28d7688e-c812-4fcc-b223-151368afe767-istio-token\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-pd5gs\" (UID: \"28d7688e-c812-4fcc-b223-151368afe767\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-pd5gs"
Oct 30 13:22:21.380763 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:21.380743    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-pd5gs"
Oct 30 13:22:21.403164 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:21.403139    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-6h9t4"
Oct 30 13:22:21.679104 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:21.679075    4943 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-85237e45-452wp"]
Oct 30 13:22:21.684056 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:21.684037    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-85237e45-452wp"
Oct 30 13:22:21.703684 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:21.703478    4943 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 13:22:21.704130 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:21.704083    4943 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-p9z57\""
Oct 30 13:22:21.704607 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:21.704586    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-85237e45-452wp"]
Oct 30 13:22:21.707515 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:21.707492    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-pd5gs"]
Oct 30 13:22:21.714692 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:21.714672    4943 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Oct 30 13:22:21.720746 ip-10-0-142-84 kubenswrapper[4943]: W1030 13:22:21.720725    4943 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod28d7688e_c812_4fcc_b223_151368afe767.slice/crio-e6a8bf040d4a4999834b7481e3b2af7326e9b4b6e4c226a125309c5d05170595 WatchSource:0}: Error finding container e6a8bf040d4a4999834b7481e3b2af7326e9b4b6e4c226a125309c5d05170595: Status 404 returned error can't find the container with id e6a8bf040d4a4999834b7481e3b2af7326e9b4b6e4c226a125309c5d05170595
Oct 30 13:22:21.723472 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:21.723437    4943 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 13:22:21.723540 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:21.723517    4943 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 13:22:21.723579 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:21.723564    4943 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 13:22:21.739429 ip-10-0-142-84 kubenswrapper[4943]: W1030 13:22:21.739404    4943 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podacf3c9ea_1f3e_40d5_88cd_2b76d8897517.slice/crio-4e06d74fb0d69f5c68a00d3e322bb7f6374612fdd257675fc3f73bb9b118414c WatchSource:0}: Error finding container 4e06d74fb0d69f5c68a00d3e322bb7f6374612fdd257675fc3f73bb9b118414c: Status 404 returned error can't find the container with id 4e06d74fb0d69f5c68a00d3e322bb7f6374612fdd257675fc3f73bb9b118414c
Oct 30 13:22:21.741935 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:21.741902    4943 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 13:22:21.742007 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:21.741980    4943 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 13:22:21.742054 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:21.742025    4943 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 13:22:21.742546 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:21.742526    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-6h9t4"]
Oct 30 13:22:21.775680 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:21.775658    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/ec130f1c-eab5-4e6a-929c-b10d5f3d9283-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-85237e45-452wp\" (UID: \"ec130f1c-eab5-4e6a-929c-b10d5f3d9283\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-85237e45-452wp"
Oct 30 13:22:21.775789 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:21.775710    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/ec130f1c-eab5-4e6a-929c-b10d5f3d9283-maas-api-xks-ca\") pod \"maas-api-revoke-sub-85237e45-452wp\" (UID: \"ec130f1c-eab5-4e6a-929c-b10d5f3d9283\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-85237e45-452wp"
Oct 30 13:22:21.775879 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:21.775816    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nl7zj\" (UniqueName: \"kubernetes.io/projected/ec130f1c-eab5-4e6a-929c-b10d5f3d9283-kube-api-access-nl7zj\") pod \"maas-api-revoke-sub-85237e45-452wp\" (UID: \"ec130f1c-eab5-4e6a-929c-b10d5f3d9283\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-85237e45-452wp"
Oct 30 13:22:21.803985 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:21.803952    4943 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Oct 30 13:22:21.877361 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:21.877315    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nl7zj\" (UniqueName: \"kubernetes.io/projected/ec130f1c-eab5-4e6a-929c-b10d5f3d9283-kube-api-access-nl7zj\") pod \"maas-api-revoke-sub-85237e45-452wp\" (UID: \"ec130f1c-eab5-4e6a-929c-b10d5f3d9283\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-85237e45-452wp"
Oct 30 13:22:21.877700 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:21.877680    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/ec130f1c-eab5-4e6a-929c-b10d5f3d9283-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-85237e45-452wp\" (UID: \"ec130f1c-eab5-4e6a-929c-b10d5f3d9283\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-85237e45-452wp"
Oct 30 13:22:21.877952 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:21.877934    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/ec130f1c-eab5-4e6a-929c-b10d5f3d9283-maas-api-xks-ca\") pod \"maas-api-revoke-sub-85237e45-452wp\" (UID: \"ec130f1c-eab5-4e6a-929c-b10d5f3d9283\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-85237e45-452wp"
Oct 30 13:22:21.878637 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:21.878430    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/ec130f1c-eab5-4e6a-929c-b10d5f3d9283-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-85237e45-452wp\" (UID: \"ec130f1c-eab5-4e6a-929c-b10d5f3d9283\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-85237e45-452wp"
Oct 30 13:22:21.880950 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:21.880899    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/ec130f1c-eab5-4e6a-929c-b10d5f3d9283-maas-api-xks-ca\") pod \"maas-api-revoke-sub-85237e45-452wp\" (UID: \"ec130f1c-eab5-4e6a-929c-b10d5f3d9283\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-85237e45-452wp"
Oct 30 13:22:21.903150 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:21.903127    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nl7zj\" (UniqueName: \"kubernetes.io/projected/ec130f1c-eab5-4e6a-929c-b10d5f3d9283-kube-api-access-nl7zj\") pod \"maas-api-revoke-sub-85237e45-452wp\" (UID: \"ec130f1c-eab5-4e6a-929c-b10d5f3d9283\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-85237e45-452wp"
Oct 30 13:22:21.997557 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:21.997484    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-85237e45-452wp"
Oct 30 13:22:22.349464 ip-10-0-142-84 kubenswrapper[4943]: W1030 13:22:22.349430    4943 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podec130f1c_eab5_4e6a_929c_b10d5f3d9283.slice/crio-1b464aa11fb75a516d8c4488a464fed48935b480288dda4223afcb26bc96417a WatchSource:0}: Error finding container 1b464aa11fb75a516d8c4488a464fed48935b480288dda4223afcb26bc96417a: Status 404 returned error can't find the container with id 1b464aa11fb75a516d8c4488a464fed48935b480288dda4223afcb26bc96417a
Oct 30 13:22:22.351157 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:22.351137    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-85237e45-452wp"]
Oct 30 13:22:22.432722 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:22.432688    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-pd5gs" event={"ID":"28d7688e-c812-4fcc-b223-151368afe767","Type":"ContainerStarted","Data":"c2ea5fd9167d5f62257af2c8b088a96935f2de64527cc69155eed73c02633fc6"}
Oct 30 13:22:22.432885 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:22.432776    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-pd5gs" event={"ID":"28d7688e-c812-4fcc-b223-151368afe767","Type":"ContainerStarted","Data":"e6a8bf040d4a4999834b7481e3b2af7326e9b4b6e4c226a125309c5d05170595"}
Oct 30 13:22:22.435158 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:22.435131    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-6h9t4" event={"ID":"acf3c9ea-1f3e-40d5-88cd-2b76d8897517","Type":"ContainerStarted","Data":"5b917183faf75df89f208e6d23a90615443ffea090d4ed8d6b13b68ec8ef840d"}
Oct 30 13:22:22.435296 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:22.435173    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-6h9t4" event={"ID":"acf3c9ea-1f3e-40d5-88cd-2b76d8897517","Type":"ContainerStarted","Data":"4e06d74fb0d69f5c68a00d3e322bb7f6374612fdd257675fc3f73bb9b118414c"}
Oct 30 13:22:22.436443 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:22.436421    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-85237e45-452wp" event={"ID":"ec130f1c-eab5-4e6a-929c-b10d5f3d9283","Type":"ContainerStarted","Data":"1b464aa11fb75a516d8c4488a464fed48935b480288dda4223afcb26bc96417a"}
Oct 30 13:22:22.509811 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:22.509711    4943 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-pd5gs" podStartSLOduration=1.509689185 podStartE2EDuration="1.509689185s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:22:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:22:22.490599033 +0000 UTC m=+1518.278602764" watchObservedRunningTime="2026-10-30 13:22:22.509689185 +0000 UTC m=+1518.297692915"
Oct 30 13:22:22.545489 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:22.545430    4943 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-6h9t4" podStartSLOduration=1.5454159939999998 podStartE2EDuration="1.545415994s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:22:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:22:22.543787889 +0000 UTC m=+1518.331791639" watchObservedRunningTime="2026-10-30 13:22:22.545415994 +0000 UTC m=+1518.333419725"
Oct 30 13:22:23.394285 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:23.393102    4943 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-pd5gs"
Oct 30 13:22:23.394285 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:23.393225    4943 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-c9ffffc7f-pd5gs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.132:15021/healthz/ready\": dial tcp 10.132.0.132:15021: connect: connection refused" start-of-body=
Oct 30 13:22:23.394285 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:23.393284    4943 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-pd5gs" podUID="28d7688e-c812-4fcc-b223-151368afe767" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.132:15021/healthz/ready\": dial tcp 10.132.0.132:15021: connect: connection refused"
Oct 30 13:22:23.405816 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:23.405789    4943 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-6h9t4"
Oct 30 13:22:23.406051 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:23.405929    4943 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-c9ffffc7f-6h9t4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.133:15021/healthz/ready\": dial tcp 10.132.0.133:15021: connect: connection refused" start-of-body=
Oct 30 13:22:23.406051 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:23.405988    4943 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-6h9t4" podUID="acf3c9ea-1f3e-40d5-88cd-2b76d8897517" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.133:15021/healthz/ready\": dial tcp 10.132.0.133:15021: connect: connection refused"
Oct 30 13:22:23.449112 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:23.449074    4943 generic.go:356] "Generic (PLEG): container finished" podID="ec130f1c-eab5-4e6a-929c-b10d5f3d9283" containerID="99f14419fcbd92fe06ba0113c75001df18a562b180cc336d217ea1dbf34c3c81" exitCode=0
Oct 30 13:22:23.449349 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:23.449322    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-85237e45-452wp" event={"ID":"ec130f1c-eab5-4e6a-929c-b10d5f3d9283","Type":"ContainerDied","Data":"99f14419fcbd92fe06ba0113c75001df18a562b180cc336d217ea1dbf34c3c81"}
Oct 30 13:22:24.383287 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:24.382698    4943 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-c9ffffc7f-pd5gs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.132:15021/healthz/ready\": dial tcp 10.132.0.132:15021: connect: connection refused" start-of-body=
Oct 30 13:22:24.383287 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:24.382756    4943 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-pd5gs" podUID="28d7688e-c812-4fcc-b223-151368afe767" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.132:15021/healthz/ready\": dial tcp 10.132.0.132:15021: connect: connection refused"
Oct 30 13:22:24.404283 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:24.403756    4943 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-c9ffffc7f-6h9t4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.133:15021/healthz/ready\": dial tcp 10.132.0.133:15021: connect: connection refused" start-of-body=
Oct 30 13:22:24.404283 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:24.403809    4943 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-6h9t4" podUID="acf3c9ea-1f3e-40d5-88cd-2b76d8897517" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.133:15021/healthz/ready\": dial tcp 10.132.0.133:15021: connect: connection refused"
Oct 30 13:22:24.722458 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:24.722428    4943 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-85237e45-452wp"
Oct 30 13:22:24.739118 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:24.739090    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/ec130f1c-eab5-4e6a-929c-b10d5f3d9283-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/ec130f1c-eab5-4e6a-929c-b10d5f3d9283-maas-api-xks-ca\") pod \"ec130f1c-eab5-4e6a-929c-b10d5f3d9283\" (UID: \"ec130f1c-eab5-4e6a-929c-b10d5f3d9283\") "
Oct 30 13:22:24.739279 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:24.739191    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ec130f1c-eab5-4e6a-929c-b10d5f3d9283-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/ec130f1c-eab5-4e6a-929c-b10d5f3d9283-maas-api-openshift-ca\") pod \"ec130f1c-eab5-4e6a-929c-b10d5f3d9283\" (UID: \"ec130f1c-eab5-4e6a-929c-b10d5f3d9283\") "
Oct 30 13:22:24.739345 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:24.739290    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ec130f1c-eab5-4e6a-929c-b10d5f3d9283-kube-api-access-nl7zj\" (UniqueName: \"kubernetes.io/projected/ec130f1c-eab5-4e6a-929c-b10d5f3d9283-kube-api-access-nl7zj\") pod \"ec130f1c-eab5-4e6a-929c-b10d5f3d9283\" (UID: \"ec130f1c-eab5-4e6a-929c-b10d5f3d9283\") "
Oct 30 13:22:24.739808 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:24.739782    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ec130f1c-eab5-4e6a-929c-b10d5f3d9283-maas-api-openshift-ca" pod "ec130f1c-eab5-4e6a-929c-b10d5f3d9283" (UID: "ec130f1c-eab5-4e6a-929c-b10d5f3d9283"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 13:22:24.740993 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:24.740958    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ec130f1c-eab5-4e6a-929c-b10d5f3d9283-maas-api-xks-ca" pod "ec130f1c-eab5-4e6a-929c-b10d5f3d9283" (UID: "ec130f1c-eab5-4e6a-929c-b10d5f3d9283"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 13:22:24.741588 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:24.741566    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ec130f1c-eab5-4e6a-929c-b10d5f3d9283-kube-api-access-nl7zj" pod "ec130f1c-eab5-4e6a-929c-b10d5f3d9283" (UID: "ec130f1c-eab5-4e6a-929c-b10d5f3d9283"). InnerVolumeSpecName "kube-api-access-nl7zj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:22:24.840770 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:24.840727    4943 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nl7zj\" (UniqueName: \"kubernetes.io/projected/ec130f1c-eab5-4e6a-929c-b10d5f3d9283-kube-api-access-nl7zj\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:22:24.840980 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:24.840965    4943 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/ec130f1c-eab5-4e6a-929c-b10d5f3d9283-maas-api-xks-ca\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:22:24.841044 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:24.841037    4943 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/ec130f1c-eab5-4e6a-929c-b10d5f3d9283-maas-api-openshift-ca\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:22:25.398286 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:25.397291    4943 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-c9ffffc7f-pd5gs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.132:15021/healthz/ready\": dial tcp 10.132.0.132:15021: connect: connection refused" start-of-body=
Oct 30 13:22:25.398286 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:25.397356    4943 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-pd5gs" podUID="28d7688e-c812-4fcc-b223-151368afe767" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.132:15021/healthz/ready\": dial tcp 10.132.0.132:15021: connect: connection refused"
Oct 30 13:22:25.407652 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:25.407622    4943 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-c9ffffc7f-6h9t4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.133:15021/healthz/ready\": dial tcp 10.132.0.133:15021: connect: connection refused" start-of-body=
Oct 30 13:22:25.408189 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:25.408154    4943 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-6h9t4" podUID="acf3c9ea-1f3e-40d5-88cd-2b76d8897517" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.133:15021/healthz/ready\": dial tcp 10.132.0.133:15021: connect: connection refused"
Oct 30 13:22:25.512956 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:25.512928    4943 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-85237e45-452wp"
Oct 30 13:22:25.513305 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:25.512927    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-85237e45-452wp" event={"ID":"ec130f1c-eab5-4e6a-929c-b10d5f3d9283","Type":"ContainerDied","Data":"1b464aa11fb75a516d8c4488a464fed48935b480288dda4223afcb26bc96417a"}
Oct 30 13:22:25.513445 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:25.513433    4943 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1b464aa11fb75a516d8c4488a464fed48935b480288dda4223afcb26bc96417a"
Oct 30 13:22:26.383285 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:26.382676    4943 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-c9ffffc7f-pd5gs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.132:15021/healthz/ready\": dial tcp 10.132.0.132:15021: connect: connection refused" start-of-body=
Oct 30 13:22:26.383285 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:26.382738    4943 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-pd5gs" podUID="28d7688e-c812-4fcc-b223-151368afe767" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.132:15021/healthz/ready\": dial tcp 10.132.0.132:15021: connect: connection refused"
Oct 30 13:22:26.408295 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:26.407654    4943 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-c9ffffc7f-6h9t4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.133:15021/healthz/ready\": dial tcp 10.132.0.133:15021: connect: connection refused" start-of-body=
Oct 30 13:22:26.408295 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:26.407716    4943 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-6h9t4" podUID="acf3c9ea-1f3e-40d5-88cd-2b76d8897517" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.133:15021/healthz/ready\": dial tcp 10.132.0.133:15021: connect: connection refused"
Oct 30 13:22:27.384718 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:27.384674    4943 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-c9ffffc7f-pd5gs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.132:15021/healthz/ready\": dial tcp 10.132.0.132:15021: connect: connection refused" start-of-body=
Oct 30 13:22:27.385036 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:27.385001    4943 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-pd5gs" podUID="28d7688e-c812-4fcc-b223-151368afe767" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.132:15021/healthz/ready\": dial tcp 10.132.0.132:15021: connect: connection refused"
Oct 30 13:22:27.411458 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:27.411426    4943 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-c9ffffc7f-6h9t4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.133:15021/healthz/ready\": dial tcp 10.132.0.133:15021: connect: connection refused" start-of-body=
Oct 30 13:22:27.412064 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:27.412019    4943 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-6h9t4" podUID="acf3c9ea-1f3e-40d5-88cd-2b76d8897517" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.133:15021/healthz/ready\": dial tcp 10.132.0.133:15021: connect: connection refused"
Oct 30 13:22:28.381410 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:28.381367    4943 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-c9ffffc7f-pd5gs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.132:15021/healthz/ready\": dial tcp 10.132.0.132:15021: connect: connection refused" start-of-body=
Oct 30 13:22:28.381588 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:28.381449    4943 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-pd5gs" podUID="28d7688e-c812-4fcc-b223-151368afe767" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.132:15021/healthz/ready\": dial tcp 10.132.0.132:15021: connect: connection refused"
Oct 30 13:22:28.403862 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:28.403827    4943 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-c9ffffc7f-6h9t4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.133:15021/healthz/ready\": dial tcp 10.132.0.133:15021: connect: connection refused" start-of-body=
Oct 30 13:22:28.403973 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:28.403880    4943 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-6h9t4" podUID="acf3c9ea-1f3e-40d5-88cd-2b76d8897517" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.133:15021/healthz/ready\": dial tcp 10.132.0.133:15021: connect: connection refused"
Oct 30 13:22:30.382408 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:30.382362    4943 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-c9ffffc7f-pd5gs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.132:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 13:22:30.382842 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:30.382429    4943 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-pd5gs" podUID="28d7688e-c812-4fcc-b223-151368afe767" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.132:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:22:30.405356 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:30.405320    4943 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-c9ffffc7f-6h9t4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.133:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 13:22:30.405537 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:30.405378    4943 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-6h9t4" podUID="acf3c9ea-1f3e-40d5-88cd-2b76d8897517" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.133:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:22:31.424504 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:31.424458    4943 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-c9ffffc7f-pd5gs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.132:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 13:22:31.425047 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:31.424532    4943 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-pd5gs" podUID="28d7688e-c812-4fcc-b223-151368afe767" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.132:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:22:31.447402 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:31.447368    4943 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-c9ffffc7f-6h9t4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.133:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 13:22:31.447562 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:31.447419    4943 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-6h9t4" podUID="acf3c9ea-1f3e-40d5-88cd-2b76d8897517" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.133:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:22:32.465454 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:32.465400    4943 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-c9ffffc7f-pd5gs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.132:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 13:22:32.465967 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:32.465495    4943 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-pd5gs" podUID="28d7688e-c812-4fcc-b223-151368afe767" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.132:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:22:32.489433 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:32.489381    4943 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-c9ffffc7f-6h9t4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.133:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 13:22:32.489609 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:32.489441    4943 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-6h9t4" podUID="acf3c9ea-1f3e-40d5-88cd-2b76d8897517" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.133:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:22:32.562140 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:32.562108    4943 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-pd5gs"
Oct 30 13:22:32.562663 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:32.562636    4943 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-pd5gs"
Oct 30 13:22:32.563502 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:32.563481    4943 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-pd5gs"
Oct 30 13:22:32.752744 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:32.752653    4943 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-6h9t4"
Oct 30 13:22:32.753136 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:32.753112    4943 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-6h9t4"
Oct 30 13:22:32.799021 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:32.798983    4943 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-6h9t4"
Oct 30 13:22:35.956385 ip-10-0-142-84 kubenswrapper[4943]: E1030 13:22:35.956345    4943 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-knqz6" containerName="extractor"
Oct 30 13:22:46.033744 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:46.033705    4943 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-9789ebd6-tlflc"]
Oct 30 13:22:46.047435 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:46.047406    4943 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-9789ebd6-tlflc"]
Oct 30 13:22:46.356344 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:46.356244    4943 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1860f3ec-4b40-4a11-9bd0-b26bcf796b90" path="/var/lib/kubelet/pods/1860f3ec-4b40-4a11-9bd0-b26bcf796b90/volumes"
Oct 30 13:22:55.813905 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:55.813870    4943 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-5444d7fdb7-jhc74"]
Oct 30 13:22:55.814472 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:55.814452    4943 cpu_manager.go:414] "containerMap: removing container" podUID="ec130f1c-eab5-4e6a-929c-b10d5f3d9283" containerName="revoke-keys"
Oct 30 13:22:55.814527 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:55.814476    4943 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ec130f1c-eab5-4e6a-929c-b10d5f3d9283" containerName="revoke-keys"
Oct 30 13:22:55.814681 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:55.814618    4943 memory_manager.go:365] "RemoveStaleState removing state" podUID="ec130f1c-eab5-4e6a-929c-b10d5f3d9283" containerName="revoke-keys"
Oct 30 13:22:55.818208 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:55.818188    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5444d7fdb7-jhc74"
Oct 30 13:22:55.849415 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:55.849338    4943 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-kvhzq\""
Oct 30 13:22:55.849540 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:55.849364    4943 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Oct 30 13:22:55.849608 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:55.849555    4943 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Oct 30 13:22:55.849696 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:55.849675    4943 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Oct 30 13:22:55.855130 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:55.855102    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5444d7fdb7-jhc74"]
Oct 30 13:22:55.961830 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:55.961799    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0d971100-57cf-4513-9ca7-1f9536f3d842-webhook-cert\") pod \"maas-controller-5444d7fdb7-jhc74\" (UID: \"0d971100-57cf-4513-9ca7-1f9536f3d842\") " pod="opendatahub/maas-controller-5444d7fdb7-jhc74"
Oct 30 13:22:55.961975 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:55.961848    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vjgpp\" (UniqueName: \"kubernetes.io/projected/0d971100-57cf-4513-9ca7-1f9536f3d842-kube-api-access-vjgpp\") pod \"maas-controller-5444d7fdb7-jhc74\" (UID: \"0d971100-57cf-4513-9ca7-1f9536f3d842\") " pod="opendatahub/maas-controller-5444d7fdb7-jhc74"
Oct 30 13:22:55.961975 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:55.961961    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0d971100-57cf-4513-9ca7-1f9536f3d842-metrics-certs\") pod \"maas-controller-5444d7fdb7-jhc74\" (UID: \"0d971100-57cf-4513-9ca7-1f9536f3d842\") " pod="opendatahub/maas-controller-5444d7fdb7-jhc74"
Oct 30 13:22:56.063176 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:56.063149    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vjgpp\" (UniqueName: \"kubernetes.io/projected/0d971100-57cf-4513-9ca7-1f9536f3d842-kube-api-access-vjgpp\") pod \"maas-controller-5444d7fdb7-jhc74\" (UID: \"0d971100-57cf-4513-9ca7-1f9536f3d842\") " pod="opendatahub/maas-controller-5444d7fdb7-jhc74"
Oct 30 13:22:56.063322 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:56.063202    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0d971100-57cf-4513-9ca7-1f9536f3d842-metrics-certs\") pod \"maas-controller-5444d7fdb7-jhc74\" (UID: \"0d971100-57cf-4513-9ca7-1f9536f3d842\") " pod="opendatahub/maas-controller-5444d7fdb7-jhc74"
Oct 30 13:22:56.063322 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:56.063292    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0d971100-57cf-4513-9ca7-1f9536f3d842-webhook-cert\") pod \"maas-controller-5444d7fdb7-jhc74\" (UID: \"0d971100-57cf-4513-9ca7-1f9536f3d842\") " pod="opendatahub/maas-controller-5444d7fdb7-jhc74"
Oct 30 13:22:56.065740 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:56.065687    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0d971100-57cf-4513-9ca7-1f9536f3d842-webhook-cert\") pod \"maas-controller-5444d7fdb7-jhc74\" (UID: \"0d971100-57cf-4513-9ca7-1f9536f3d842\") " pod="opendatahub/maas-controller-5444d7fdb7-jhc74"
Oct 30 13:22:56.065740 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:56.065713    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0d971100-57cf-4513-9ca7-1f9536f3d842-metrics-certs\") pod \"maas-controller-5444d7fdb7-jhc74\" (UID: \"0d971100-57cf-4513-9ca7-1f9536f3d842\") " pod="opendatahub/maas-controller-5444d7fdb7-jhc74"
Oct 30 13:22:56.088749 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:56.088725    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vjgpp\" (UniqueName: \"kubernetes.io/projected/0d971100-57cf-4513-9ca7-1f9536f3d842-kube-api-access-vjgpp\") pod \"maas-controller-5444d7fdb7-jhc74\" (UID: \"0d971100-57cf-4513-9ca7-1f9536f3d842\") " pod="opendatahub/maas-controller-5444d7fdb7-jhc74"
Oct 30 13:22:56.134055 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:56.134030    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5444d7fdb7-jhc74"
Oct 30 13:22:56.442515 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:56.442486    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5444d7fdb7-jhc74"]
Oct 30 13:22:56.442635 ip-10-0-142-84 kubenswrapper[4943]: W1030 13:22:56.442486    4943 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0d971100_57cf_4513_9ca7_1f9536f3d842.slice/crio-ffacad5051df22b23820ed750acfe704a781b352368ef313358a650cf045e690 WatchSource:0}: Error finding container ffacad5051df22b23820ed750acfe704a781b352368ef313358a650cf045e690: Status 404 returned error can't find the container with id ffacad5051df22b23820ed750acfe704a781b352368ef313358a650cf045e690
Oct 30 13:22:56.443653 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:56.443635    4943 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 13:22:56.731308 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:56.731215    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5444d7fdb7-jhc74" event={"ID":"0d971100-57cf-4513-9ca7-1f9536f3d842","Type":"ContainerStarted","Data":"ffacad5051df22b23820ed750acfe704a781b352368ef313358a650cf045e690"}
Oct 30 13:22:58.746782 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:58.746745    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5444d7fdb7-jhc74" event={"ID":"0d971100-57cf-4513-9ca7-1f9536f3d842","Type":"ContainerStarted","Data":"41b8bc31f89d008f1357d1c432c3a57b9da2abe78541b8911ad5f71e8410044b"}
Oct 30 13:22:58.747199 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:58.746883    4943 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-5444d7fdb7-jhc74"
Oct 30 13:22:58.799875 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:22:58.799829    4943 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/maas-controller-5444d7fdb7-jhc74" podStartSLOduration=2.251482359 podStartE2EDuration="3.799816731s" totalImagesPullingTime="1.548334372s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:22:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:22:58.797789665 +0000 UTC m=+1554.585793395" watchObservedRunningTime="2026-10-30 13:22:58.799816731 +0000 UTC m=+1554.587820630"
Oct 30 13:23:05.954897 ip-10-0-142-84 kubenswrapper[4943]: E1030 13:23:05.954858    4943 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-knqz6" containerName="extractor"
Oct 30 13:23:07.628944 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:23:07.628914    4943 scope.go:122] "RemoveContainer" containerID="44ec6a14c6426331b705f0e4714d5f0bfcb896e1313d09ca5ef7930ab3c0b11d"
Oct 30 13:23:09.038381 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:23:09.038349    4943 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-76f0e08b-t78dl"]
Oct 30 13:23:09.051974 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:23:09.051945    4943 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-76f0e08b-t78dl"]
Oct 30 13:23:09.756896 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:23:09.756864    4943 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-5444d7fdb7-jhc74"
Oct 30 13:23:10.349530 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:23:10.349499    4943 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="07b7a046-d95c-4344-bbf9-4abb90b36fca" path="/var/lib/kubelet/pods/07b7a046-d95c-4344-bbf9-4abb90b36fca/volumes"
Oct 30 13:23:17.070440 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:23:17.070353    4943 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-bc53a85a-dg5hj"]
Oct 30 13:23:17.083987 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:23:17.083957    4943 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-bc53a85a-dg5hj"]
Oct 30 13:23:18.350492 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:23:18.350457    4943 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b8812677-29e8-49cd-af4f-9dbb6e3658a8" path="/var/lib/kubelet/pods/b8812677-29e8-49cd-af4f-9dbb6e3658a8/volumes"
Oct 30 13:23:30.480015 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:23:30.479983    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-hdmrm"]
Oct 30 13:23:33.077404 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:23:33.077358    4943 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-62f1bc5f-klfbt"]
Oct 30 13:23:33.100927 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:23:33.100891    4943 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-62f1bc5f-klfbt"]
Oct 30 13:23:33.608527 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:23:33.608487    4943 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-c9ffffc7f-pd5gs container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.132:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 13:23:33.608921 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:23:33.608880    4943 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-pd5gs" podUID="28d7688e-c812-4fcc-b223-151368afe767" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.132:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:23:34.350220 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:23:34.350186    4943 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6918acda-77b9-4c03-86f0-c9ba259f80ae" path="/var/lib/kubelet/pods/6918acda-77b9-4c03-86f0-c9ba259f80ae/volumes"
Oct 30 13:23:35.957491 ip-10-0-142-84 kubenswrapper[4943]: E1030 13:23:35.957448    4943 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-knqz6" containerName="extractor"
Oct 30 13:23:36.237165 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:23:36.237086    4943 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-c1e271a9-g4m2m"]
Oct 30 13:23:36.242603 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:23:36.242583    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c1e271a9-g4m2m"
Oct 30 13:23:36.261013 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:23:36.260984    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-c1e271a9-g4m2m"]
Oct 30 13:23:36.271929 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:23:36.271886    4943 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 13:23:36.271929 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:23:36.271915    4943 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Oct 30 13:23:36.271929 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:23:36.271900    4943 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Oct 30 13:23:36.278228 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:23:36.278205    4943 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-p9z57\""
Oct 30 13:23:36.380864 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:23:36.380824    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/143e6e24-96e1-47bb-ba3a-1aa21b7bc699-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-c1e271a9-g4m2m\" (UID: \"143e6e24-96e1-47bb-ba3a-1aa21b7bc699\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c1e271a9-g4m2m"
Oct 30 13:23:36.381054 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:23:36.380898    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j5xsl\" (UniqueName: \"kubernetes.io/projected/143e6e24-96e1-47bb-ba3a-1aa21b7bc699-kube-api-access-j5xsl\") pod \"maas-api-revoke-sub-c1e271a9-g4m2m\" (UID: \"143e6e24-96e1-47bb-ba3a-1aa21b7bc699\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c1e271a9-g4m2m"
Oct 30 13:23:36.381054 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:23:36.380956    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/143e6e24-96e1-47bb-ba3a-1aa21b7bc699-maas-api-xks-ca\") pod \"maas-api-revoke-sub-c1e271a9-g4m2m\" (UID: \"143e6e24-96e1-47bb-ba3a-1aa21b7bc699\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c1e271a9-g4m2m"
Oct 30 13:23:36.482431 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:23:36.482387    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/143e6e24-96e1-47bb-ba3a-1aa21b7bc699-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-c1e271a9-g4m2m\" (UID: \"143e6e24-96e1-47bb-ba3a-1aa21b7bc699\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c1e271a9-g4m2m"
Oct 30 13:23:36.482641 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:23:36.482458    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j5xsl\" (UniqueName: \"kubernetes.io/projected/143e6e24-96e1-47bb-ba3a-1aa21b7bc699-kube-api-access-j5xsl\") pod \"maas-api-revoke-sub-c1e271a9-g4m2m\" (UID: \"143e6e24-96e1-47bb-ba3a-1aa21b7bc699\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c1e271a9-g4m2m"
Oct 30 13:23:36.482641 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:23:36.482586    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/143e6e24-96e1-47bb-ba3a-1aa21b7bc699-maas-api-xks-ca\") pod \"maas-api-revoke-sub-c1e271a9-g4m2m\" (UID: \"143e6e24-96e1-47bb-ba3a-1aa21b7bc699\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c1e271a9-g4m2m"
Oct 30 13:23:36.483138 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:23:36.483116    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/143e6e24-96e1-47bb-ba3a-1aa21b7bc699-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-c1e271a9-g4m2m\" (UID: \"143e6e24-96e1-47bb-ba3a-1aa21b7bc699\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c1e271a9-g4m2m"
Oct 30 13:23:36.484868 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:23:36.484840    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/143e6e24-96e1-47bb-ba3a-1aa21b7bc699-maas-api-xks-ca\") pod \"maas-api-revoke-sub-c1e271a9-g4m2m\" (UID: \"143e6e24-96e1-47bb-ba3a-1aa21b7bc699\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c1e271a9-g4m2m"
Oct 30 13:23:36.527013 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:23:36.526942    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j5xsl\" (UniqueName: \"kubernetes.io/projected/143e6e24-96e1-47bb-ba3a-1aa21b7bc699-kube-api-access-j5xsl\") pod \"maas-api-revoke-sub-c1e271a9-g4m2m\" (UID: \"143e6e24-96e1-47bb-ba3a-1aa21b7bc699\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c1e271a9-g4m2m"
Oct 30 13:23:36.556779 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:23:36.556746    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c1e271a9-g4m2m"
Oct 30 13:23:36.860210 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:23:36.860179    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-c1e271a9-g4m2m"]
Oct 30 13:23:36.860811 ip-10-0-142-84 kubenswrapper[4943]: W1030 13:23:36.860778    4943 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod143e6e24_96e1_47bb_ba3a_1aa21b7bc699.slice/crio-399b504b61c20408e0087b49bfdb0b861770767e26b1eef87a081922ca8a0b07 WatchSource:0}: Error finding container 399b504b61c20408e0087b49bfdb0b861770767e26b1eef87a081922ca8a0b07: Status 404 returned error can't find the container with id 399b504b61c20408e0087b49bfdb0b861770767e26b1eef87a081922ca8a0b07
Oct 30 13:23:37.053044 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:23:37.053008    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c1e271a9-g4m2m" event={"ID":"143e6e24-96e1-47bb-ba3a-1aa21b7bc699","Type":"ContainerStarted","Data":"399b504b61c20408e0087b49bfdb0b861770767e26b1eef87a081922ca8a0b07"}
Oct 30 13:23:38.061640 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:23:38.061539    4943 generic.go:356] "Generic (PLEG): container finished" podID="143e6e24-96e1-47bb-ba3a-1aa21b7bc699" containerID="d5cb7effea101c94b4ee77e35f11234b54ddf8a2768619f121bac8476da1a522" exitCode=0
Oct 30 13:23:38.061640 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:23:38.061591    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c1e271a9-g4m2m" event={"ID":"143e6e24-96e1-47bb-ba3a-1aa21b7bc699","Type":"ContainerDied","Data":"d5cb7effea101c94b4ee77e35f11234b54ddf8a2768619f121bac8476da1a522"}
Oct 30 13:23:39.229273 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:23:39.229231    4943 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c1e271a9-g4m2m"
Oct 30 13:23:39.308927 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:23:39.308890    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/143e6e24-96e1-47bb-ba3a-1aa21b7bc699-kube-api-access-j5xsl\" (UniqueName: \"kubernetes.io/projected/143e6e24-96e1-47bb-ba3a-1aa21b7bc699-kube-api-access-j5xsl\") pod \"143e6e24-96e1-47bb-ba3a-1aa21b7bc699\" (UID: \"143e6e24-96e1-47bb-ba3a-1aa21b7bc699\") "
Oct 30 13:23:39.309100 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:23:39.309083    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/143e6e24-96e1-47bb-ba3a-1aa21b7bc699-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/143e6e24-96e1-47bb-ba3a-1aa21b7bc699-maas-api-openshift-ca\") pod \"143e6e24-96e1-47bb-ba3a-1aa21b7bc699\" (UID: \"143e6e24-96e1-47bb-ba3a-1aa21b7bc699\") "
Oct 30 13:23:39.309182 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:23:39.309165    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/143e6e24-96e1-47bb-ba3a-1aa21b7bc699-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/143e6e24-96e1-47bb-ba3a-1aa21b7bc699-maas-api-xks-ca\") pod \"143e6e24-96e1-47bb-ba3a-1aa21b7bc699\" (UID: \"143e6e24-96e1-47bb-ba3a-1aa21b7bc699\") "
Oct 30 13:23:39.309566 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:23:39.309529    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/143e6e24-96e1-47bb-ba3a-1aa21b7bc699-maas-api-openshift-ca" pod "143e6e24-96e1-47bb-ba3a-1aa21b7bc699" (UID: "143e6e24-96e1-47bb-ba3a-1aa21b7bc699"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 13:23:39.311077 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:23:39.311051    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/143e6e24-96e1-47bb-ba3a-1aa21b7bc699-kube-api-access-j5xsl" pod "143e6e24-96e1-47bb-ba3a-1aa21b7bc699" (UID: "143e6e24-96e1-47bb-ba3a-1aa21b7bc699"). InnerVolumeSpecName "kube-api-access-j5xsl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:23:39.311242 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:23:39.311219    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/143e6e24-96e1-47bb-ba3a-1aa21b7bc699-maas-api-xks-ca" pod "143e6e24-96e1-47bb-ba3a-1aa21b7bc699" (UID: "143e6e24-96e1-47bb-ba3a-1aa21b7bc699"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 13:23:39.410526 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:23:39.410441    4943 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/143e6e24-96e1-47bb-ba3a-1aa21b7bc699-maas-api-openshift-ca\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:23:39.410526 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:23:39.410476    4943 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/143e6e24-96e1-47bb-ba3a-1aa21b7bc699-maas-api-xks-ca\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:23:39.410526 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:23:39.410486    4943 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j5xsl\" (UniqueName: \"kubernetes.io/projected/143e6e24-96e1-47bb-ba3a-1aa21b7bc699-kube-api-access-j5xsl\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:23:40.077050 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:23:40.077018    4943 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c1e271a9-g4m2m"
Oct 30 13:23:40.077234 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:23:40.077048    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c1e271a9-g4m2m" event={"ID":"143e6e24-96e1-47bb-ba3a-1aa21b7bc699","Type":"ContainerDied","Data":"399b504b61c20408e0087b49bfdb0b861770767e26b1eef87a081922ca8a0b07"}
Oct 30 13:23:40.077234 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:23:40.077101    4943 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="399b504b61c20408e0087b49bfdb0b861770767e26b1eef87a081922ca8a0b07"
Oct 30 13:23:47.486679 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:23:47.486641    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-hdmrm"]
Oct 30 13:23:51.985726 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:23:51.985686    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-hdmrm"]
Oct 30 13:23:53.037600 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:23:53.037558    4943 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-438019a3-qqgbp"]
Oct 30 13:23:53.051451 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:23:53.051403    4943 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-438019a3-qqgbp"]
Oct 30 13:23:54.367446 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:23:54.366841    4943 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b1b6f388-523f-4ec1-bf0d-4d31e1e6a6c6" path="/var/lib/kubelet/pods/b1b6f388-523f-4ec1-bf0d-4d31e1e6a6c6/volumes"
Oct 30 13:23:56.198074 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:23:56.198037    4943 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-2e1f5b5f-j5bfx"]
Oct 30 13:23:56.198640 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:23:56.198619    4943 cpu_manager.go:414] "containerMap: removing container" podUID="143e6e24-96e1-47bb-ba3a-1aa21b7bc699" containerName="revoke-keys"
Oct 30 13:23:56.198640 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:23:56.198636    4943 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="143e6e24-96e1-47bb-ba3a-1aa21b7bc699" containerName="revoke-keys"
Oct 30 13:23:56.198791 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:23:56.198707    4943 memory_manager.go:365] "RemoveStaleState removing state" podUID="143e6e24-96e1-47bb-ba3a-1aa21b7bc699" containerName="revoke-keys"
Oct 30 13:23:56.202107 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:23:56.202085    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-2e1f5b5f-j5bfx"
Oct 30 13:23:56.222034 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:23:56.222002    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-2e1f5b5f-j5bfx"]
Oct 30 13:23:56.232434 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:23:56.232401    4943 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 13:23:56.232650 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:23:56.232417    4943 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Oct 30 13:23:56.232650 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:23:56.232485    4943 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-p9z57\""
Oct 30 13:23:56.233780 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:23:56.233761    4943 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Oct 30 13:23:56.301685 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:23:56.301644    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/b84d7af2-77e6-4d59-844c-e8a9ab7279f8-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-2e1f5b5f-j5bfx\" (UID: \"b84d7af2-77e6-4d59-844c-e8a9ab7279f8\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-2e1f5b5f-j5bfx"
Oct 30 13:23:56.301866 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:23:56.301702    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8pqwd\" (UniqueName: \"kubernetes.io/projected/b84d7af2-77e6-4d59-844c-e8a9ab7279f8-kube-api-access-8pqwd\") pod \"maas-api-revoke-sub-2e1f5b5f-j5bfx\" (UID: \"b84d7af2-77e6-4d59-844c-e8a9ab7279f8\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-2e1f5b5f-j5bfx"
Oct 30 13:23:56.301866 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:23:56.301801    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/b84d7af2-77e6-4d59-844c-e8a9ab7279f8-maas-api-xks-ca\") pod \"maas-api-revoke-sub-2e1f5b5f-j5bfx\" (UID: \"b84d7af2-77e6-4d59-844c-e8a9ab7279f8\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-2e1f5b5f-j5bfx"
Oct 30 13:23:56.403271 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:23:56.403228    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/b84d7af2-77e6-4d59-844c-e8a9ab7279f8-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-2e1f5b5f-j5bfx\" (UID: \"b84d7af2-77e6-4d59-844c-e8a9ab7279f8\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-2e1f5b5f-j5bfx"
Oct 30 13:23:56.403446 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:23:56.403320    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8pqwd\" (UniqueName: \"kubernetes.io/projected/b84d7af2-77e6-4d59-844c-e8a9ab7279f8-kube-api-access-8pqwd\") pod \"maas-api-revoke-sub-2e1f5b5f-j5bfx\" (UID: \"b84d7af2-77e6-4d59-844c-e8a9ab7279f8\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-2e1f5b5f-j5bfx"
Oct 30 13:23:56.403446 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:23:56.403352    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/b84d7af2-77e6-4d59-844c-e8a9ab7279f8-maas-api-xks-ca\") pod \"maas-api-revoke-sub-2e1f5b5f-j5bfx\" (UID: \"b84d7af2-77e6-4d59-844c-e8a9ab7279f8\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-2e1f5b5f-j5bfx"
Oct 30 13:23:56.403903 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:23:56.403875    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/b84d7af2-77e6-4d59-844c-e8a9ab7279f8-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-2e1f5b5f-j5bfx\" (UID: \"b84d7af2-77e6-4d59-844c-e8a9ab7279f8\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-2e1f5b5f-j5bfx"
Oct 30 13:23:56.405580 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:23:56.405555    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/b84d7af2-77e6-4d59-844c-e8a9ab7279f8-maas-api-xks-ca\") pod \"maas-api-revoke-sub-2e1f5b5f-j5bfx\" (UID: \"b84d7af2-77e6-4d59-844c-e8a9ab7279f8\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-2e1f5b5f-j5bfx"
Oct 30 13:23:56.428489 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:23:56.428462    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8pqwd\" (UniqueName: \"kubernetes.io/projected/b84d7af2-77e6-4d59-844c-e8a9ab7279f8-kube-api-access-8pqwd\") pod \"maas-api-revoke-sub-2e1f5b5f-j5bfx\" (UID: \"b84d7af2-77e6-4d59-844c-e8a9ab7279f8\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-2e1f5b5f-j5bfx"
Oct 30 13:23:56.534049 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:23:56.533959    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-2e1f5b5f-j5bfx"
Oct 30 13:23:56.839981 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:23:56.839948    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-2e1f5b5f-j5bfx"]
Oct 30 13:23:56.840731 ip-10-0-142-84 kubenswrapper[4943]: W1030 13:23:56.840705    4943 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb84d7af2_77e6_4d59_844c_e8a9ab7279f8.slice/crio-ba973e6ec378f3401e86d7c17ac33ba9c52c565cc6054ff348ea47890381d3d0 WatchSource:0}: Error finding container ba973e6ec378f3401e86d7c17ac33ba9c52c565cc6054ff348ea47890381d3d0: Status 404 returned error can't find the container with id ba973e6ec378f3401e86d7c17ac33ba9c52c565cc6054ff348ea47890381d3d0
Oct 30 13:23:57.212355 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:23:57.212310    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-2e1f5b5f-j5bfx" event={"ID":"b84d7af2-77e6-4d59-844c-e8a9ab7279f8","Type":"ContainerStarted","Data":"7ca1e6087058579c298fc9d7ffb917e2e3e2973732b161626f2ef324e0cb09d3"}
Oct 30 13:23:57.212757 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:23:57.212380    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-2e1f5b5f-j5bfx" event={"ID":"b84d7af2-77e6-4d59-844c-e8a9ab7279f8","Type":"ContainerStarted","Data":"ba973e6ec378f3401e86d7c17ac33ba9c52c565cc6054ff348ea47890381d3d0"}
Oct 30 13:23:57.258628 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:23:57.258572    4943 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-2e1f5b5f-j5bfx" podStartSLOduration=1.258557369 podStartE2EDuration="1.258557369s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:23:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:23:57.2561146 +0000 UTC m=+1613.044118330" watchObservedRunningTime="2026-10-30 13:23:57.258557369 +0000 UTC m=+1613.046561098"
Oct 30 13:23:58.221235 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:23:58.221199    4943 generic.go:356] "Generic (PLEG): container finished" podID="b84d7af2-77e6-4d59-844c-e8a9ab7279f8" containerID="7ca1e6087058579c298fc9d7ffb917e2e3e2973732b161626f2ef324e0cb09d3" exitCode=0
Oct 30 13:23:58.221677 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:23:58.221253    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-2e1f5b5f-j5bfx" event={"ID":"b84d7af2-77e6-4d59-844c-e8a9ab7279f8","Type":"ContainerDied","Data":"7ca1e6087058579c298fc9d7ffb917e2e3e2973732b161626f2ef324e0cb09d3"}
Oct 30 13:23:59.376568 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:23:59.376540    4943 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-2e1f5b5f-j5bfx"
Oct 30 13:23:59.540136 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:23:59.540041    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b84d7af2-77e6-4d59-844c-e8a9ab7279f8-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/b84d7af2-77e6-4d59-844c-e8a9ab7279f8-maas-api-openshift-ca\") pod \"b84d7af2-77e6-4d59-844c-e8a9ab7279f8\" (UID: \"b84d7af2-77e6-4d59-844c-e8a9ab7279f8\") "
Oct 30 13:23:59.540136 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:23:59.540095    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/b84d7af2-77e6-4d59-844c-e8a9ab7279f8-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/b84d7af2-77e6-4d59-844c-e8a9ab7279f8-maas-api-xks-ca\") pod \"b84d7af2-77e6-4d59-844c-e8a9ab7279f8\" (UID: \"b84d7af2-77e6-4d59-844c-e8a9ab7279f8\") "
Oct 30 13:23:59.540419 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:23:59.540250    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b84d7af2-77e6-4d59-844c-e8a9ab7279f8-kube-api-access-8pqwd\" (UniqueName: \"kubernetes.io/projected/b84d7af2-77e6-4d59-844c-e8a9ab7279f8-kube-api-access-8pqwd\") pod \"b84d7af2-77e6-4d59-844c-e8a9ab7279f8\" (UID: \"b84d7af2-77e6-4d59-844c-e8a9ab7279f8\") "
Oct 30 13:23:59.540491 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:23:59.540443    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b84d7af2-77e6-4d59-844c-e8a9ab7279f8-maas-api-openshift-ca" pod "b84d7af2-77e6-4d59-844c-e8a9ab7279f8" (UID: "b84d7af2-77e6-4d59-844c-e8a9ab7279f8"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 13:23:59.540618 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:23:59.540596    4943 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/b84d7af2-77e6-4d59-844c-e8a9ab7279f8-maas-api-openshift-ca\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:23:59.542332 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:23:59.542302    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b84d7af2-77e6-4d59-844c-e8a9ab7279f8-maas-api-xks-ca" pod "b84d7af2-77e6-4d59-844c-e8a9ab7279f8" (UID: "b84d7af2-77e6-4d59-844c-e8a9ab7279f8"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 13:23:59.542418 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:23:59.542349    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b84d7af2-77e6-4d59-844c-e8a9ab7279f8-kube-api-access-8pqwd" pod "b84d7af2-77e6-4d59-844c-e8a9ab7279f8" (UID: "b84d7af2-77e6-4d59-844c-e8a9ab7279f8"). InnerVolumeSpecName "kube-api-access-8pqwd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:23:59.641880 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:23:59.641835    4943 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8pqwd\" (UniqueName: \"kubernetes.io/projected/b84d7af2-77e6-4d59-844c-e8a9ab7279f8-kube-api-access-8pqwd\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:23:59.641880 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:23:59.641871    4943 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/b84d7af2-77e6-4d59-844c-e8a9ab7279f8-maas-api-xks-ca\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:24:00.236426 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:24:00.236391    4943 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-2e1f5b5f-j5bfx"
Oct 30 13:24:00.236624 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:24:00.236395    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-2e1f5b5f-j5bfx" event={"ID":"b84d7af2-77e6-4d59-844c-e8a9ab7279f8","Type":"ContainerDied","Data":"ba973e6ec378f3401e86d7c17ac33ba9c52c565cc6054ff348ea47890381d3d0"}
Oct 30 13:24:00.236624 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:24:00.236535    4943 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ba973e6ec378f3401e86d7c17ac33ba9c52c565cc6054ff348ea47890381d3d0"
Oct 30 13:24:04.038753 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:24:04.038706    4943 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-1699bfc5-9crvx"]
Oct 30 13:24:04.052693 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:24:04.052658    4943 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-1699bfc5-9crvx"]
Oct 30 13:24:04.351292 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:24:04.351207    4943 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f7d7355b-c38e-49dd-8cd0-c5dd66306a90" path="/var/lib/kubelet/pods/f7d7355b-c38e-49dd-8cd0-c5dd66306a90/volumes"
Oct 30 13:24:05.955194 ip-10-0-142-84 kubenswrapper[4943]: E1030 13:24:05.955152    4943 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-knqz6" containerName="extractor"
Oct 30 13:24:07.474552 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:24:07.474517    4943 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-ec843018-6qx99"]
Oct 30 13:24:07.474986 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:24:07.474974    4943 cpu_manager.go:414] "containerMap: removing container" podUID="b84d7af2-77e6-4d59-844c-e8a9ab7279f8" containerName="revoke-keys"
Oct 30 13:24:07.474986 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:24:07.474986    4943 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b84d7af2-77e6-4d59-844c-e8a9ab7279f8" containerName="revoke-keys"
Oct 30 13:24:07.475076 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:24:07.475064    4943 memory_manager.go:365] "RemoveStaleState removing state" podUID="b84d7af2-77e6-4d59-844c-e8a9ab7279f8" containerName="revoke-keys"
Oct 30 13:24:07.479665 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:24:07.479642    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ec843018-6qx99"
Oct 30 13:24:07.512671 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:24:07.512641    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/f7b09437-8214-40b9-a2be-ad764125bd09-maas-api-xks-ca\") pod \"maas-api-revoke-sub-ec843018-6qx99\" (UID: \"f7b09437-8214-40b9-a2be-ad764125bd09\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ec843018-6qx99"
Oct 30 13:24:07.512861 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:24:07.512681    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8pzbv\" (UniqueName: \"kubernetes.io/projected/f7b09437-8214-40b9-a2be-ad764125bd09-kube-api-access-8pzbv\") pod \"maas-api-revoke-sub-ec843018-6qx99\" (UID: \"f7b09437-8214-40b9-a2be-ad764125bd09\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ec843018-6qx99"
Oct 30 13:24:07.512861 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:24:07.512708    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/f7b09437-8214-40b9-a2be-ad764125bd09-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-ec843018-6qx99\" (UID: \"f7b09437-8214-40b9-a2be-ad764125bd09\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ec843018-6qx99"
Oct 30 13:24:07.513248 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:24:07.513211    4943 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 13:24:07.513401 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:24:07.513290    4943 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-p9z57\""
Oct 30 13:24:07.516939 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:24:07.516914    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-ec843018-6qx99"]
Oct 30 13:24:07.536491 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:24:07.536461    4943 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Oct 30 13:24:07.536656 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:24:07.536599    4943 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Oct 30 13:24:07.580869 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:24:07.580826    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-hdmrm"]
Oct 30 13:24:07.614251 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:24:07.614218    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/f7b09437-8214-40b9-a2be-ad764125bd09-maas-api-xks-ca\") pod \"maas-api-revoke-sub-ec843018-6qx99\" (UID: \"f7b09437-8214-40b9-a2be-ad764125bd09\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ec843018-6qx99"
Oct 30 13:24:07.614439 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:24:07.614257    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8pzbv\" (UniqueName: \"kubernetes.io/projected/f7b09437-8214-40b9-a2be-ad764125bd09-kube-api-access-8pzbv\") pod \"maas-api-revoke-sub-ec843018-6qx99\" (UID: \"f7b09437-8214-40b9-a2be-ad764125bd09\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ec843018-6qx99"
Oct 30 13:24:07.614439 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:24:07.614317    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/f7b09437-8214-40b9-a2be-ad764125bd09-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-ec843018-6qx99\" (UID: \"f7b09437-8214-40b9-a2be-ad764125bd09\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ec843018-6qx99"
Oct 30 13:24:07.615047 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:24:07.615024    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/f7b09437-8214-40b9-a2be-ad764125bd09-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-ec843018-6qx99\" (UID: \"f7b09437-8214-40b9-a2be-ad764125bd09\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ec843018-6qx99"
Oct 30 13:24:07.616590 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:24:07.616572    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/f7b09437-8214-40b9-a2be-ad764125bd09-maas-api-xks-ca\") pod \"maas-api-revoke-sub-ec843018-6qx99\" (UID: \"f7b09437-8214-40b9-a2be-ad764125bd09\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ec843018-6qx99"
Oct 30 13:24:07.639772 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:24:07.639745    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8pzbv\" (UniqueName: \"kubernetes.io/projected/f7b09437-8214-40b9-a2be-ad764125bd09-kube-api-access-8pzbv\") pod \"maas-api-revoke-sub-ec843018-6qx99\" (UID: \"f7b09437-8214-40b9-a2be-ad764125bd09\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ec843018-6qx99"
Oct 30 13:24:07.654294 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:24:07.654251    4943 scope.go:122] "RemoveContainer" containerID="b3cc5cfb534fb865001cbd1e0ab1e850f63c1142c25d6e5b6a199af6d7b8f740"
Oct 30 13:24:07.664055 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:24:07.664033    4943 scope.go:122] "RemoveContainer" containerID="504f6f77769160d06b2691fc88b5fb1fb4be437ec6185b98f75df665abe250b4"
Oct 30 13:24:07.793022 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:24:07.792925    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ec843018-6qx99"
Oct 30 13:24:08.098018 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:24:08.097985    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-ec843018-6qx99"]
Oct 30 13:24:08.098806 ip-10-0-142-84 kubenswrapper[4943]: W1030 13:24:08.098774    4943 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf7b09437_8214_40b9_a2be_ad764125bd09.slice/crio-d0c2eab69ab94e4151687c2f4eb9dbe85e566893ad5aff7db93b358172e05001 WatchSource:0}: Error finding container d0c2eab69ab94e4151687c2f4eb9dbe85e566893ad5aff7db93b358172e05001: Status 404 returned error can't find the container with id d0c2eab69ab94e4151687c2f4eb9dbe85e566893ad5aff7db93b358172e05001
Oct 30 13:24:08.291568 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:24:08.291534    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ec843018-6qx99" event={"ID":"f7b09437-8214-40b9-a2be-ad764125bd09","Type":"ContainerStarted","Data":"d0c2eab69ab94e4151687c2f4eb9dbe85e566893ad5aff7db93b358172e05001"}
Oct 30 13:24:09.299893 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:24:09.299857    4943 generic.go:356] "Generic (PLEG): container finished" podID="f7b09437-8214-40b9-a2be-ad764125bd09" containerID="b2d6b337254c7cf43711f3ff3680686fb6fa0cfbf37be56e52491f0be86fa161" exitCode=0
Oct 30 13:24:09.300323 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:24:09.299906    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ec843018-6qx99" event={"ID":"f7b09437-8214-40b9-a2be-ad764125bd09","Type":"ContainerDied","Data":"b2d6b337254c7cf43711f3ff3680686fb6fa0cfbf37be56e52491f0be86fa161"}
Oct 30 13:24:10.450919 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:24:10.450894    4943 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ec843018-6qx99"
Oct 30 13:24:10.546182 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:24:10.546149    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f7b09437-8214-40b9-a2be-ad764125bd09-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/f7b09437-8214-40b9-a2be-ad764125bd09-maas-api-openshift-ca\") pod \"f7b09437-8214-40b9-a2be-ad764125bd09\" (UID: \"f7b09437-8214-40b9-a2be-ad764125bd09\") "
Oct 30 13:24:10.546182 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:24:10.546187    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/f7b09437-8214-40b9-a2be-ad764125bd09-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/f7b09437-8214-40b9-a2be-ad764125bd09-maas-api-xks-ca\") pod \"f7b09437-8214-40b9-a2be-ad764125bd09\" (UID: \"f7b09437-8214-40b9-a2be-ad764125bd09\") "
Oct 30 13:24:10.546427 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:24:10.546230    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f7b09437-8214-40b9-a2be-ad764125bd09-kube-api-access-8pzbv\" (UniqueName: \"kubernetes.io/projected/f7b09437-8214-40b9-a2be-ad764125bd09-kube-api-access-8pzbv\") pod \"f7b09437-8214-40b9-a2be-ad764125bd09\" (UID: \"f7b09437-8214-40b9-a2be-ad764125bd09\") "
Oct 30 13:24:10.546524 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:24:10.546502    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f7b09437-8214-40b9-a2be-ad764125bd09-maas-api-openshift-ca" pod "f7b09437-8214-40b9-a2be-ad764125bd09" (UID: "f7b09437-8214-40b9-a2be-ad764125bd09"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 13:24:10.548409 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:24:10.548383    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f7b09437-8214-40b9-a2be-ad764125bd09-kube-api-access-8pzbv" pod "f7b09437-8214-40b9-a2be-ad764125bd09" (UID: "f7b09437-8214-40b9-a2be-ad764125bd09"). InnerVolumeSpecName "kube-api-access-8pzbv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:24:10.548512 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:24:10.548401    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f7b09437-8214-40b9-a2be-ad764125bd09-maas-api-xks-ca" pod "f7b09437-8214-40b9-a2be-ad764125bd09" (UID: "f7b09437-8214-40b9-a2be-ad764125bd09"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 13:24:10.647335 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:24:10.647223    4943 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8pzbv\" (UniqueName: \"kubernetes.io/projected/f7b09437-8214-40b9-a2be-ad764125bd09-kube-api-access-8pzbv\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:24:10.647335 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:24:10.647254    4943 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/f7b09437-8214-40b9-a2be-ad764125bd09-maas-api-openshift-ca\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:24:10.647335 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:24:10.647291    4943 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/f7b09437-8214-40b9-a2be-ad764125bd09-maas-api-xks-ca\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:24:11.061293 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:24:11.061225    4943 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-201d684c-rh5w4"]
Oct 30 13:24:11.076182 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:24:11.076148    4943 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-201d684c-rh5w4"]
Oct 30 13:24:11.314298 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:24:11.314192    4943 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ec843018-6qx99"
Oct 30 13:24:11.314298 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:24:11.314227    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ec843018-6qx99" event={"ID":"f7b09437-8214-40b9-a2be-ad764125bd09","Type":"ContainerDied","Data":"d0c2eab69ab94e4151687c2f4eb9dbe85e566893ad5aff7db93b358172e05001"}
Oct 30 13:24:11.314487 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:24:11.314308    4943 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d0c2eab69ab94e4151687c2f4eb9dbe85e566893ad5aff7db93b358172e05001"
Oct 30 13:24:12.349302 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:24:12.349248    4943 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6f0595d0-f37c-4067-b1e0-de986b38918b" path="/var/lib/kubelet/pods/6f0595d0-f37c-4067-b1e0-de986b38918b/volumes"
Oct 30 13:24:15.096057 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:24:15.096016    4943 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-acd1d13c-sbxnv"]
Oct 30 13:24:15.110137 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:24:15.110103    4943 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-acd1d13c-sbxnv"]
Oct 30 13:24:16.349351 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:24:16.349310    4943 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0786b50b-b6e4-4788-bcd0-6fc3463e4d72" path="/var/lib/kubelet/pods/0786b50b-b6e4-4788-bcd0-6fc3463e4d72/volumes"
Oct 30 13:24:33.352256 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:24:33.352220    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-hdmrm"]
Oct 30 13:24:36.044284 ip-10-0-142-84 kubenswrapper[4943]: E1030 13:24:36.037497    4943 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-knqz6" containerName="extractor"
Oct 30 13:24:37.034676 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:24:37.034637    4943 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-06c73c5c-lzbxz"]
Oct 30 13:24:37.048278 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:24:37.048229    4943 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-06c73c5c-lzbxz"]
Oct 30 13:24:38.349923 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:24:38.349889    4943 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4ffb2c71-a66d-4f4f-8fdd-48707259c348" path="/var/lib/kubelet/pods/4ffb2c71-a66d-4f4f-8fdd-48707259c348/volumes"
Oct 30 13:24:45.516440 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:24:45.516362    4943 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-046e1343-g8p7z"]
Oct 30 13:24:45.516887 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:24:45.516869    4943 cpu_manager.go:414] "containerMap: removing container" podUID="f7b09437-8214-40b9-a2be-ad764125bd09" containerName="revoke-keys"
Oct 30 13:24:45.516943 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:24:45.516888    4943 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f7b09437-8214-40b9-a2be-ad764125bd09" containerName="revoke-keys"
Oct 30 13:24:45.516986 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:24:45.516980    4943 memory_manager.go:365] "RemoveStaleState removing state" podUID="f7b09437-8214-40b9-a2be-ad764125bd09" containerName="revoke-keys"
Oct 30 13:24:45.521290 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:24:45.521256    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-046e1343-g8p7z"
Oct 30 13:24:45.552228 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:24:45.552196    4943 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Oct 30 13:24:45.552418 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:24:45.552361    4943 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Oct 30 13:24:45.552418 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:24:45.552380    4943 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 13:24:45.552630 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:24:45.552605    4943 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-p9z57\""
Oct 30 13:24:45.555576 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:24:45.555551    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-046e1343-g8p7z"]
Oct 30 13:24:45.569095 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:24:45.569070    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k9ts9\" (UniqueName: \"kubernetes.io/projected/29e6fafd-e405-4c80-bde1-3c76ae91e50f-kube-api-access-k9ts9\") pod \"maas-api-revoke-sub-046e1343-g8p7z\" (UID: \"29e6fafd-e405-4c80-bde1-3c76ae91e50f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-046e1343-g8p7z"
Oct 30 13:24:45.569182 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:24:45.569132    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/29e6fafd-e405-4c80-bde1-3c76ae91e50f-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-046e1343-g8p7z\" (UID: \"29e6fafd-e405-4c80-bde1-3c76ae91e50f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-046e1343-g8p7z"
Oct 30 13:24:45.569281 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:24:45.569227    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/29e6fafd-e405-4c80-bde1-3c76ae91e50f-maas-api-xks-ca\") pod \"maas-api-revoke-sub-046e1343-g8p7z\" (UID: \"29e6fafd-e405-4c80-bde1-3c76ae91e50f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-046e1343-g8p7z"
Oct 30 13:24:45.670301 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:24:45.670234    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k9ts9\" (UniqueName: \"kubernetes.io/projected/29e6fafd-e405-4c80-bde1-3c76ae91e50f-kube-api-access-k9ts9\") pod \"maas-api-revoke-sub-046e1343-g8p7z\" (UID: \"29e6fafd-e405-4c80-bde1-3c76ae91e50f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-046e1343-g8p7z"
Oct 30 13:24:45.670474 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:24:45.670328    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/29e6fafd-e405-4c80-bde1-3c76ae91e50f-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-046e1343-g8p7z\" (UID: \"29e6fafd-e405-4c80-bde1-3c76ae91e50f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-046e1343-g8p7z"
Oct 30 13:24:45.670474 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:24:45.670366    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/29e6fafd-e405-4c80-bde1-3c76ae91e50f-maas-api-xks-ca\") pod \"maas-api-revoke-sub-046e1343-g8p7z\" (UID: \"29e6fafd-e405-4c80-bde1-3c76ae91e50f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-046e1343-g8p7z"
Oct 30 13:24:45.671021 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:24:45.670990    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/29e6fafd-e405-4c80-bde1-3c76ae91e50f-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-046e1343-g8p7z\" (UID: \"29e6fafd-e405-4c80-bde1-3c76ae91e50f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-046e1343-g8p7z"
Oct 30 13:24:45.672759 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:24:45.672737    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/29e6fafd-e405-4c80-bde1-3c76ae91e50f-maas-api-xks-ca\") pod \"maas-api-revoke-sub-046e1343-g8p7z\" (UID: \"29e6fafd-e405-4c80-bde1-3c76ae91e50f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-046e1343-g8p7z"
Oct 30 13:24:45.696097 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:24:45.696071    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k9ts9\" (UniqueName: \"kubernetes.io/projected/29e6fafd-e405-4c80-bde1-3c76ae91e50f-kube-api-access-k9ts9\") pod \"maas-api-revoke-sub-046e1343-g8p7z\" (UID: \"29e6fafd-e405-4c80-bde1-3c76ae91e50f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-046e1343-g8p7z"
Oct 30 13:24:45.834832 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:24:45.834748    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-046e1343-g8p7z"
Oct 30 13:24:46.140671 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:24:46.140640    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-046e1343-g8p7z"]
Oct 30 13:24:46.141301 ip-10-0-142-84 kubenswrapper[4943]: W1030 13:24:46.141272    4943 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod29e6fafd_e405_4c80_bde1_3c76ae91e50f.slice/crio-c2981e3d7fd1e6741651ed03b28eb459f61b4a304e366f66159ce7177e9da628 WatchSource:0}: Error finding container c2981e3d7fd1e6741651ed03b28eb459f61b4a304e366f66159ce7177e9da628: Status 404 returned error can't find the container with id c2981e3d7fd1e6741651ed03b28eb459f61b4a304e366f66159ce7177e9da628
Oct 30 13:24:46.588321 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:24:46.588283    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-046e1343-g8p7z" event={"ID":"29e6fafd-e405-4c80-bde1-3c76ae91e50f","Type":"ContainerStarted","Data":"ae735397781c16b3b664b9b6353de739c3d4ec1dfb93097e09911dcd3f69b30b"}
Oct 30 13:24:46.588703 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:24:46.588367    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-046e1343-g8p7z" event={"ID":"29e6fafd-e405-4c80-bde1-3c76ae91e50f","Type":"ContainerStarted","Data":"c2981e3d7fd1e6741651ed03b28eb459f61b4a304e366f66159ce7177e9da628"}
Oct 30 13:24:46.640093 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:24:46.640028    4943 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-046e1343-g8p7z" podStartSLOduration=1.6400135150000001 podStartE2EDuration="1.640013515s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:24:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:24:46.635547973 +0000 UTC m=+1662.423551703" watchObservedRunningTime="2026-10-30 13:24:46.640013515 +0000 UTC m=+1662.428017242"
Oct 30 13:24:47.596833 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:24:47.596797    4943 generic.go:356] "Generic (PLEG): container finished" podID="29e6fafd-e405-4c80-bde1-3c76ae91e50f" containerID="ae735397781c16b3b664b9b6353de739c3d4ec1dfb93097e09911dcd3f69b30b" exitCode=0
Oct 30 13:24:47.597310 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:24:47.596883    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-046e1343-g8p7z" event={"ID":"29e6fafd-e405-4c80-bde1-3c76ae91e50f","Type":"ContainerDied","Data":"ae735397781c16b3b664b9b6353de739c3d4ec1dfb93097e09911dcd3f69b30b"}
Oct 30 13:24:48.751053 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:24:48.751028    4943 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-046e1343-g8p7z"
Oct 30 13:24:48.801510 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:24:48.801470    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/29e6fafd-e405-4c80-bde1-3c76ae91e50f-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/29e6fafd-e405-4c80-bde1-3c76ae91e50f-maas-api-xks-ca\") pod \"29e6fafd-e405-4c80-bde1-3c76ae91e50f\" (UID: \"29e6fafd-e405-4c80-bde1-3c76ae91e50f\") "
Oct 30 13:24:48.801675 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:24:48.801548    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/29e6fafd-e405-4c80-bde1-3c76ae91e50f-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/29e6fafd-e405-4c80-bde1-3c76ae91e50f-maas-api-openshift-ca\") pod \"29e6fafd-e405-4c80-bde1-3c76ae91e50f\" (UID: \"29e6fafd-e405-4c80-bde1-3c76ae91e50f\") "
Oct 30 13:24:48.801761 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:24:48.801675    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/29e6fafd-e405-4c80-bde1-3c76ae91e50f-kube-api-access-k9ts9\" (UniqueName: \"kubernetes.io/projected/29e6fafd-e405-4c80-bde1-3c76ae91e50f-kube-api-access-k9ts9\") pod \"29e6fafd-e405-4c80-bde1-3c76ae91e50f\" (UID: \"29e6fafd-e405-4c80-bde1-3c76ae91e50f\") "
Oct 30 13:24:48.802375 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:24:48.802342    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/29e6fafd-e405-4c80-bde1-3c76ae91e50f-maas-api-openshift-ca" pod "29e6fafd-e405-4c80-bde1-3c76ae91e50f" (UID: "29e6fafd-e405-4c80-bde1-3c76ae91e50f"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 13:24:48.804196 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:24:48.804168    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/29e6fafd-e405-4c80-bde1-3c76ae91e50f-maas-api-xks-ca" pod "29e6fafd-e405-4c80-bde1-3c76ae91e50f" (UID: "29e6fafd-e405-4c80-bde1-3c76ae91e50f"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 13:24:48.804576 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:24:48.804549    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/29e6fafd-e405-4c80-bde1-3c76ae91e50f-kube-api-access-k9ts9" pod "29e6fafd-e405-4c80-bde1-3c76ae91e50f" (UID: "29e6fafd-e405-4c80-bde1-3c76ae91e50f"). InnerVolumeSpecName "kube-api-access-k9ts9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:24:48.902936 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:24:48.902855    4943 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k9ts9\" (UniqueName: \"kubernetes.io/projected/29e6fafd-e405-4c80-bde1-3c76ae91e50f-kube-api-access-k9ts9\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:24:48.902936 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:24:48.902884    4943 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/29e6fafd-e405-4c80-bde1-3c76ae91e50f-maas-api-xks-ca\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:24:48.902936 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:24:48.902898    4943 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/29e6fafd-e405-4c80-bde1-3c76ae91e50f-maas-api-openshift-ca\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:24:49.611307 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:24:49.611253    4943 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-046e1343-g8p7z"
Oct 30 13:24:49.611307 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:24:49.611289    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-046e1343-g8p7z" event={"ID":"29e6fafd-e405-4c80-bde1-3c76ae91e50f","Type":"ContainerDied","Data":"c2981e3d7fd1e6741651ed03b28eb459f61b4a304e366f66159ce7177e9da628"}
Oct 30 13:24:49.611537 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:24:49.611354    4943 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c2981e3d7fd1e6741651ed03b28eb459f61b4a304e366f66159ce7177e9da628"
Oct 30 13:24:50.036445 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:24:50.036407    4943 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-67237791-pb7w6"]
Oct 30 13:24:50.050658 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:24:50.050628    4943 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-c0194afd-djn2v"]
Oct 30 13:24:50.053652 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:24:50.053621    4943 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-67237791-pb7w6"]
Oct 30 13:24:50.056609 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:24:50.056583    4943 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-c0194afd-djn2v"]
Oct 30 13:24:50.350396 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:24:50.350323    4943 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1b91fb8f-779b-440c-b6a7-d2b89bf020a1" path="/var/lib/kubelet/pods/1b91fb8f-779b-440c-b6a7-d2b89bf020a1/volumes"
Oct 30 13:24:50.350738 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:24:50.350725    4943 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5a018183-4b6a-4d06-a087-34c2f5168b46" path="/var/lib/kubelet/pods/5a018183-4b6a-4d06-a087-34c2f5168b46/volumes"
Oct 30 13:24:55.848524 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:24:55.848485    4943 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-d3fba056-j59bv"]
Oct 30 13:24:55.849062 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:24:55.849044    4943 cpu_manager.go:414] "containerMap: removing container" podUID="29e6fafd-e405-4c80-bde1-3c76ae91e50f" containerName="revoke-keys"
Oct 30 13:24:55.849124 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:24:55.849063    4943 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="29e6fafd-e405-4c80-bde1-3c76ae91e50f" containerName="revoke-keys"
Oct 30 13:24:55.849167 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:24:55.849157    4943 memory_manager.go:365] "RemoveStaleState removing state" podUID="29e6fafd-e405-4c80-bde1-3c76ae91e50f" containerName="revoke-keys"
Oct 30 13:24:55.852388 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:24:55.852372    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d3fba056-j59bv"
Oct 30 13:24:55.869379 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:24:55.869350    4943 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 13:24:55.869587 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:24:55.869474    4943 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Oct 30 13:24:55.872026 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:24:55.871998    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-d3fba056-j59bv"]
Oct 30 13:24:55.884471 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:24:55.884446    4943 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Oct 30 13:24:55.884613 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:24:55.884545    4943 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-p9z57\""
Oct 30 13:24:55.970807 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:24:55.970770    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/f50b6e2c-85e9-49b5-87e2-83377054f324-maas-api-xks-ca\") pod \"maas-api-revoke-sub-d3fba056-j59bv\" (UID: \"f50b6e2c-85e9-49b5-87e2-83377054f324\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d3fba056-j59bv"
Oct 30 13:24:55.971015 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:24:55.970833    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2q9zg\" (UniqueName: \"kubernetes.io/projected/f50b6e2c-85e9-49b5-87e2-83377054f324-kube-api-access-2q9zg\") pod \"maas-api-revoke-sub-d3fba056-j59bv\" (UID: \"f50b6e2c-85e9-49b5-87e2-83377054f324\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d3fba056-j59bv"
Oct 30 13:24:55.971015 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:24:55.970947    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/f50b6e2c-85e9-49b5-87e2-83377054f324-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-d3fba056-j59bv\" (UID: \"f50b6e2c-85e9-49b5-87e2-83377054f324\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d3fba056-j59bv"
Oct 30 13:24:56.071734 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:24:56.071690    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/f50b6e2c-85e9-49b5-87e2-83377054f324-maas-api-xks-ca\") pod \"maas-api-revoke-sub-d3fba056-j59bv\" (UID: \"f50b6e2c-85e9-49b5-87e2-83377054f324\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d3fba056-j59bv"
Oct 30 13:24:56.071946 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:24:56.071763    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2q9zg\" (UniqueName: \"kubernetes.io/projected/f50b6e2c-85e9-49b5-87e2-83377054f324-kube-api-access-2q9zg\") pod \"maas-api-revoke-sub-d3fba056-j59bv\" (UID: \"f50b6e2c-85e9-49b5-87e2-83377054f324\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d3fba056-j59bv"
Oct 30 13:24:56.071946 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:24:56.071790    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/f50b6e2c-85e9-49b5-87e2-83377054f324-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-d3fba056-j59bv\" (UID: \"f50b6e2c-85e9-49b5-87e2-83377054f324\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d3fba056-j59bv"
Oct 30 13:24:56.072558 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:24:56.072535    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/f50b6e2c-85e9-49b5-87e2-83377054f324-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-d3fba056-j59bv\" (UID: \"f50b6e2c-85e9-49b5-87e2-83377054f324\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d3fba056-j59bv"
Oct 30 13:24:56.074133 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:24:56.074108    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/f50b6e2c-85e9-49b5-87e2-83377054f324-maas-api-xks-ca\") pod \"maas-api-revoke-sub-d3fba056-j59bv\" (UID: \"f50b6e2c-85e9-49b5-87e2-83377054f324\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d3fba056-j59bv"
Oct 30 13:24:56.099057 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:24:56.098996    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2q9zg\" (UniqueName: \"kubernetes.io/projected/f50b6e2c-85e9-49b5-87e2-83377054f324-kube-api-access-2q9zg\") pod \"maas-api-revoke-sub-d3fba056-j59bv\" (UID: \"f50b6e2c-85e9-49b5-87e2-83377054f324\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d3fba056-j59bv"
Oct 30 13:24:56.165465 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:24:56.165425    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d3fba056-j59bv"
Oct 30 13:24:56.472023 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:24:56.471965    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-d3fba056-j59bv"]
Oct 30 13:24:56.472294 ip-10-0-142-84 kubenswrapper[4943]: W1030 13:24:56.472256    4943 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf50b6e2c_85e9_49b5_87e2_83377054f324.slice/crio-248c65a65efc2bf28ef1973bca6eb62a4a064551a407f4828663b66a699d61e2 WatchSource:0}: Error finding container 248c65a65efc2bf28ef1973bca6eb62a4a064551a407f4828663b66a699d61e2: Status 404 returned error can't find the container with id 248c65a65efc2bf28ef1973bca6eb62a4a064551a407f4828663b66a699d61e2
Oct 30 13:24:56.659635 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:24:56.659597    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d3fba056-j59bv" event={"ID":"f50b6e2c-85e9-49b5-87e2-83377054f324","Type":"ContainerStarted","Data":"248c65a65efc2bf28ef1973bca6eb62a4a064551a407f4828663b66a699d61e2"}
Oct 30 13:24:56.717173 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:24:56.717139    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-hdmrm"]
Oct 30 13:24:57.672219 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:24:57.672124    4943 generic.go:356] "Generic (PLEG): container finished" podID="f50b6e2c-85e9-49b5-87e2-83377054f324" containerID="dfd227df1403c1380d7bf706f409bd110d6aa1d6178c21a88898de6f6b0fddca" exitCode=0
Oct 30 13:24:57.672866 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:24:57.672209    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d3fba056-j59bv" event={"ID":"f50b6e2c-85e9-49b5-87e2-83377054f324","Type":"ContainerDied","Data":"dfd227df1403c1380d7bf706f409bd110d6aa1d6178c21a88898de6f6b0fddca"}
Oct 30 13:24:59.521529 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:24:59.521486    4943 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d3fba056-j59bv"
Oct 30 13:24:59.566200 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:24:59.566165    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/f50b6e2c-85e9-49b5-87e2-83377054f324-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/f50b6e2c-85e9-49b5-87e2-83377054f324-maas-api-xks-ca\") pod \"f50b6e2c-85e9-49b5-87e2-83377054f324\" (UID: \"f50b6e2c-85e9-49b5-87e2-83377054f324\") "
Oct 30 13:24:59.566626 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:24:59.566601    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f50b6e2c-85e9-49b5-87e2-83377054f324-kube-api-access-2q9zg\" (UniqueName: \"kubernetes.io/projected/f50b6e2c-85e9-49b5-87e2-83377054f324-kube-api-access-2q9zg\") pod \"f50b6e2c-85e9-49b5-87e2-83377054f324\" (UID: \"f50b6e2c-85e9-49b5-87e2-83377054f324\") "
Oct 30 13:24:59.566922 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:24:59.566887    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f50b6e2c-85e9-49b5-87e2-83377054f324-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/f50b6e2c-85e9-49b5-87e2-83377054f324-maas-api-openshift-ca\") pod \"f50b6e2c-85e9-49b5-87e2-83377054f324\" (UID: \"f50b6e2c-85e9-49b5-87e2-83377054f324\") "
Oct 30 13:24:59.572524 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:24:59.572487    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f50b6e2c-85e9-49b5-87e2-83377054f324-maas-api-openshift-ca" pod "f50b6e2c-85e9-49b5-87e2-83377054f324" (UID: "f50b6e2c-85e9-49b5-87e2-83377054f324"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 13:24:59.590479 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:24:59.587634    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f50b6e2c-85e9-49b5-87e2-83377054f324-maas-api-xks-ca" pod "f50b6e2c-85e9-49b5-87e2-83377054f324" (UID: "f50b6e2c-85e9-49b5-87e2-83377054f324"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 13:24:59.590479 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:24:59.589706    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f50b6e2c-85e9-49b5-87e2-83377054f324-kube-api-access-2q9zg" pod "f50b6e2c-85e9-49b5-87e2-83377054f324" (UID: "f50b6e2c-85e9-49b5-87e2-83377054f324"). InnerVolumeSpecName "kube-api-access-2q9zg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:24:59.694669 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:24:59.694632    4943 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/f50b6e2c-85e9-49b5-87e2-83377054f324-maas-api-xks-ca\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:24:59.694889 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:24:59.694871    4943 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2q9zg\" (UniqueName: \"kubernetes.io/projected/f50b6e2c-85e9-49b5-87e2-83377054f324-kube-api-access-2q9zg\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:24:59.695001 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:24:59.694986    4943 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/f50b6e2c-85e9-49b5-87e2-83377054f324-maas-api-openshift-ca\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:24:59.781305 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:24:59.776694    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d3fba056-j59bv" event={"ID":"f50b6e2c-85e9-49b5-87e2-83377054f324","Type":"ContainerDied","Data":"248c65a65efc2bf28ef1973bca6eb62a4a064551a407f4828663b66a699d61e2"}
Oct 30 13:24:59.781305 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:24:59.776764    4943 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="248c65a65efc2bf28ef1973bca6eb62a4a064551a407f4828663b66a699d61e2"
Oct 30 13:24:59.781305 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:24:59.776837    4943 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d3fba056-j59bv"
Oct 30 13:25:05.955899 ip-10-0-142-84 kubenswrapper[4943]: E1030 13:25:05.955856    4943 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-knqz6" containerName="extractor"
Oct 30 13:25:07.081600 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:25:07.081563    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-hdmrm"]
Oct 30 13:25:07.696778 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:25:07.696738    4943 scope.go:122] "RemoveContainer" containerID="2c16604c945c13324aa77fc889221b8fb2abd4deeed2dae7406028f881929db7"
Oct 30 13:25:07.706759 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:25:07.706736    4943 scope.go:122] "RemoveContainer" containerID="bb01a7a097fcb1600c75cf6b3629fa2666873e502dadb07161f8bc8c12dfefd0"
Oct 30 13:25:07.715884 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:25:07.715864    4943 scope.go:122] "RemoveContainer" containerID="1f4ae2ad3ab92bcb88f71224221e4563df821638db844dffca8f7204b8275b37"
Oct 30 13:25:07.724120 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:25:07.724100    4943 scope.go:122] "RemoveContainer" containerID="577ce31b069a2df8ef798f2cba5c5b89aaf19002a7e45613e077e384c835c4ee"
Oct 30 13:25:07.733741 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:25:07.733723    4943 scope.go:122] "RemoveContainer" containerID="6c1ba9cd5851dfac47048825a8d9479ce683c6c7f2cdda92db7c5a375845c4be"
Oct 30 13:25:11.519207 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:25:11.519166    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-hdmrm"]
Oct 30 13:25:19.036170 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:25:19.036125    4943 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-fbc73eb9-j9rdq"]
Oct 30 13:25:19.049490 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:25:19.049443    4943 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-fbc73eb9-j9rdq"]
Oct 30 13:25:20.350066 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:25:20.350030    4943 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ab205844-0e5c-4c60-846c-6f6627a10692" path="/var/lib/kubelet/pods/ab205844-0e5c-4c60-846c-6f6627a10692/volumes"
Oct 30 13:25:22.161224 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:25:22.161180    4943 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-ae594979-wt4d9"]
Oct 30 13:25:22.161695 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:25:22.161680    4943 cpu_manager.go:414] "containerMap: removing container" podUID="f50b6e2c-85e9-49b5-87e2-83377054f324" containerName="revoke-keys"
Oct 30 13:25:22.161739 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:25:22.161696    4943 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f50b6e2c-85e9-49b5-87e2-83377054f324" containerName="revoke-keys"
Oct 30 13:25:22.161803 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:25:22.161794    4943 memory_manager.go:365] "RemoveStaleState removing state" podUID="f50b6e2c-85e9-49b5-87e2-83377054f324" containerName="revoke-keys"
Oct 30 13:25:22.166880 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:25:22.166858    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ae594979-wt4d9"
Oct 30 13:25:22.185739 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:25:22.185703    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/0c0cc2dc-f621-4def-b952-174d35c4e244-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-ae594979-wt4d9\" (UID: \"0c0cc2dc-f621-4def-b952-174d35c4e244\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ae594979-wt4d9"
Oct 30 13:25:22.185907 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:25:22.185772    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k5b9b\" (UniqueName: \"kubernetes.io/projected/0c0cc2dc-f621-4def-b952-174d35c4e244-kube-api-access-k5b9b\") pod \"maas-api-revoke-sub-ae594979-wt4d9\" (UID: \"0c0cc2dc-f621-4def-b952-174d35c4e244\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ae594979-wt4d9"
Oct 30 13:25:22.185907 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:25:22.185894    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/0c0cc2dc-f621-4def-b952-174d35c4e244-maas-api-xks-ca\") pod \"maas-api-revoke-sub-ae594979-wt4d9\" (UID: \"0c0cc2dc-f621-4def-b952-174d35c4e244\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ae594979-wt4d9"
Oct 30 13:25:22.186248 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:25:22.186222    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-ae594979-wt4d9"]
Oct 30 13:25:22.196605 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:25:22.196576    4943 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 13:25:22.196763 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:25:22.196636    4943 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Oct 30 13:25:22.197695 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:25:22.197676    4943 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Oct 30 13:25:22.197751 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:25:22.197683    4943 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-p9z57\""
Oct 30 13:25:22.286791 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:25:22.286742    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/0c0cc2dc-f621-4def-b952-174d35c4e244-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-ae594979-wt4d9\" (UID: \"0c0cc2dc-f621-4def-b952-174d35c4e244\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ae594979-wt4d9"
Oct 30 13:25:22.287003 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:25:22.286885    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k5b9b\" (UniqueName: \"kubernetes.io/projected/0c0cc2dc-f621-4def-b952-174d35c4e244-kube-api-access-k5b9b\") pod \"maas-api-revoke-sub-ae594979-wt4d9\" (UID: \"0c0cc2dc-f621-4def-b952-174d35c4e244\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ae594979-wt4d9"
Oct 30 13:25:22.287073 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:25:22.287019    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/0c0cc2dc-f621-4def-b952-174d35c4e244-maas-api-xks-ca\") pod \"maas-api-revoke-sub-ae594979-wt4d9\" (UID: \"0c0cc2dc-f621-4def-b952-174d35c4e244\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ae594979-wt4d9"
Oct 30 13:25:22.287569 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:25:22.287543    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/0c0cc2dc-f621-4def-b952-174d35c4e244-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-ae594979-wt4d9\" (UID: \"0c0cc2dc-f621-4def-b952-174d35c4e244\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ae594979-wt4d9"
Oct 30 13:25:22.289401 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:25:22.289378    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/0c0cc2dc-f621-4def-b952-174d35c4e244-maas-api-xks-ca\") pod \"maas-api-revoke-sub-ae594979-wt4d9\" (UID: \"0c0cc2dc-f621-4def-b952-174d35c4e244\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ae594979-wt4d9"
Oct 30 13:25:22.312707 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:25:22.312673    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k5b9b\" (UniqueName: \"kubernetes.io/projected/0c0cc2dc-f621-4def-b952-174d35c4e244-kube-api-access-k5b9b\") pod \"maas-api-revoke-sub-ae594979-wt4d9\" (UID: \"0c0cc2dc-f621-4def-b952-174d35c4e244\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ae594979-wt4d9"
Oct 30 13:25:22.480004 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:25:22.479963    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ae594979-wt4d9"
Oct 30 13:25:22.784585 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:25:22.784551    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-ae594979-wt4d9"]
Oct 30 13:25:22.785674 ip-10-0-142-84 kubenswrapper[4943]: W1030 13:25:22.785647    4943 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0c0cc2dc_f621_4def_b952_174d35c4e244.slice/crio-df79bf01c1eee7adcd7e4aa1d53545aef5ac1076fc1202f870294f478c5c70a4 WatchSource:0}: Error finding container df79bf01c1eee7adcd7e4aa1d53545aef5ac1076fc1202f870294f478c5c70a4: Status 404 returned error can't find the container with id df79bf01c1eee7adcd7e4aa1d53545aef5ac1076fc1202f870294f478c5c70a4
Oct 30 13:25:22.955245 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:25:22.955206    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ae594979-wt4d9" event={"ID":"0c0cc2dc-f621-4def-b952-174d35c4e244","Type":"ContainerStarted","Data":"df79bf01c1eee7adcd7e4aa1d53545aef5ac1076fc1202f870294f478c5c70a4"}
Oct 30 13:25:23.963525 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:25:23.963489    4943 generic.go:356] "Generic (PLEG): container finished" podID="0c0cc2dc-f621-4def-b952-174d35c4e244" containerID="d283650d4ca806731bd6a64abebcb33477b9fc6d91bfb7f61615092d6676f388" exitCode=0
Oct 30 13:25:23.963915 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:25:23.963577    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ae594979-wt4d9" event={"ID":"0c0cc2dc-f621-4def-b952-174d35c4e244","Type":"ContainerDied","Data":"d283650d4ca806731bd6a64abebcb33477b9fc6d91bfb7f61615092d6676f388"}
Oct 30 13:25:25.111157 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:25:25.111128    4943 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ae594979-wt4d9"
Oct 30 13:25:25.215473 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:25:25.215438    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0c0cc2dc-f621-4def-b952-174d35c4e244-kube-api-access-k5b9b\" (UniqueName: \"kubernetes.io/projected/0c0cc2dc-f621-4def-b952-174d35c4e244-kube-api-access-k5b9b\") pod \"0c0cc2dc-f621-4def-b952-174d35c4e244\" (UID: \"0c0cc2dc-f621-4def-b952-174d35c4e244\") "
Oct 30 13:25:25.215669 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:25:25.215572    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0c0cc2dc-f621-4def-b952-174d35c4e244-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/0c0cc2dc-f621-4def-b952-174d35c4e244-maas-api-openshift-ca\") pod \"0c0cc2dc-f621-4def-b952-174d35c4e244\" (UID: \"0c0cc2dc-f621-4def-b952-174d35c4e244\") "
Oct 30 13:25:25.215743 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:25:25.215675    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/0c0cc2dc-f621-4def-b952-174d35c4e244-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/0c0cc2dc-f621-4def-b952-174d35c4e244-maas-api-xks-ca\") pod \"0c0cc2dc-f621-4def-b952-174d35c4e244\" (UID: \"0c0cc2dc-f621-4def-b952-174d35c4e244\") "
Oct 30 13:25:25.215975 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:25:25.215946    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0c0cc2dc-f621-4def-b952-174d35c4e244-maas-api-openshift-ca" pod "0c0cc2dc-f621-4def-b952-174d35c4e244" (UID: "0c0cc2dc-f621-4def-b952-174d35c4e244"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 13:25:25.217698 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:25:25.217673    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0c0cc2dc-f621-4def-b952-174d35c4e244-maas-api-xks-ca" pod "0c0cc2dc-f621-4def-b952-174d35c4e244" (UID: "0c0cc2dc-f621-4def-b952-174d35c4e244"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 13:25:25.217786 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:25:25.217738    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0c0cc2dc-f621-4def-b952-174d35c4e244-kube-api-access-k5b9b" pod "0c0cc2dc-f621-4def-b952-174d35c4e244" (UID: "0c0cc2dc-f621-4def-b952-174d35c4e244"). InnerVolumeSpecName "kube-api-access-k5b9b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:25:25.317008 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:25:25.316968    4943 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k5b9b\" (UniqueName: \"kubernetes.io/projected/0c0cc2dc-f621-4def-b952-174d35c4e244-kube-api-access-k5b9b\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:25:25.317008 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:25:25.317003    4943 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/0c0cc2dc-f621-4def-b952-174d35c4e244-maas-api-openshift-ca\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:25:25.317008 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:25:25.317015    4943 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/0c0cc2dc-f621-4def-b952-174d35c4e244-maas-api-xks-ca\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:25:25.977402 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:25:25.977364    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ae594979-wt4d9" event={"ID":"0c0cc2dc-f621-4def-b952-174d35c4e244","Type":"ContainerDied","Data":"df79bf01c1eee7adcd7e4aa1d53545aef5ac1076fc1202f870294f478c5c70a4"}
Oct 30 13:25:25.977402 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:25:25.977400    4943 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ae594979-wt4d9"
Oct 30 13:25:25.977620 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:25:25.977416    4943 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="df79bf01c1eee7adcd7e4aa1d53545aef5ac1076fc1202f870294f478c5c70a4"
Oct 30 13:25:33.377640 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:25:33.377597    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-hdmrm"]
Oct 30 13:25:35.956911 ip-10-0-142-84 kubenswrapper[4943]: E1030 13:25:35.956873    4943 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-knqz6" containerName="extractor"
Oct 30 13:25:38.430296 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:25:38.429951    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-hdmrm"]
Oct 30 13:25:54.999119 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:25:54.999085    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-hdmrm"]
Oct 30 13:25:59.047987 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:25:59.047949    4943 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-e413f738-h9kh9"]
Oct 30 13:25:59.061429 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:25:59.061396    4943 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-e413f738-h9kh9"]
Oct 30 13:25:59.316469 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:25:59.316385    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-hdmrm"]
Oct 30 13:26:00.353969 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:26:00.353922    4943 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="52f8160d-f86d-495e-9e3c-710353788068" path="/var/lib/kubelet/pods/52f8160d-f86d-495e-9e3c-710353788068/volumes"
Oct 30 13:26:05.956046 ip-10-0-142-84 kubenswrapper[4943]: E1030 13:26:05.956000    4943 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-knqz6" containerName="extractor"
Oct 30 13:26:07.043340 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:26:07.043291    4943 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-c0df5bf0-2l5v5"]
Oct 30 13:26:07.057523 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:26:07.057485    4943 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-c0df5bf0-2l5v5"]
Oct 30 13:26:07.780231 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:26:07.780191    4943 scope.go:122] "RemoveContainer" containerID="0bac164e5acfb9089fa1f0bd80606e742e958b61beca4f3118f1f3af2545b0c8"
Oct 30 13:26:07.790475 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:26:07.790447    4943 scope.go:122] "RemoveContainer" containerID="d7bec3d261ca1f991075b9db2facb3d5dd77b621b06514b896c8e78c7f8d19ff"
Oct 30 13:26:07.803966 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:26:07.803939    4943 scope.go:122] "RemoveContainer" containerID="d1a4c0870cf8aa8e49e38a17eda5935a5f79583f119a89cd0b5416740b54140c"
Oct 30 13:26:07.813133 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:26:07.813108    4943 scope.go:122] "RemoveContainer" containerID="a961027a259c994b5f534c18a8147c7b1bdde4e2ec7a8e5ce1b2409914390c73"
Oct 30 13:26:07.821845 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:26:07.821823    4943 scope.go:122] "RemoveContainer" containerID="bd34232f1d63e999d6cd1d3fb1395fddfe7a0610320ec2bb7799d7ee9092f793"
Oct 30 13:26:07.831076 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:26:07.831052    4943 scope.go:122] "RemoveContainer" containerID="09f9ec76060fad5a818fee4d217d80149fdc85091c1a08150d9c387862920924"
Oct 30 13:26:07.839801 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:26:07.839776    4943 scope.go:122] "RemoveContainer" containerID="be9685d42c019f2312e664f602fa3280d036aeca530b9ec2d548663634e8e347"
Oct 30 13:26:08.349588 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:26:08.349554    4943 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="bd28f696-44c5-49ef-ac4b-9b63e9b7e5bb" path="/var/lib/kubelet/pods/bd28f696-44c5-49ef-ac4b-9b63e9b7e5bb/volumes"
Oct 30 13:26:12.042847 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:26:12.042807    4943 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-f462c87d-t6b8r"]
Oct 30 13:26:12.056799 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:26:12.056761    4943 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-f462c87d-t6b8r"]
Oct 30 13:26:12.349486 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:26:12.349403    4943 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2b3fefeb-d946-48e8-afab-773935b7a4e7" path="/var/lib/kubelet/pods/2b3fefeb-d946-48e8-afab-773935b7a4e7/volumes"
Oct 30 13:26:15.986797 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:26:15.986710    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-hdmrm"]
Oct 30 13:26:18.611288 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:26:18.609485    4943 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-c9ffffc7f-pd5gs container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.132:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 13:26:18.611288 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:26:18.609590    4943 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-pd5gs" podUID="28d7688e-c812-4fcc-b223-151368afe767" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.132:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:26:18.807433 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:26:18.807385    4943 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-c9ffffc7f-6h9t4 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.133:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 13:26:18.807769 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:26:18.807719    4943 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-6h9t4" podUID="acf3c9ea-1f3e-40d5-88cd-2b76d8897517" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.133:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:26:33.185919 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:26:33.185876    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-hdmrm"]
Oct 30 13:26:35.045203 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:26:35.045168    4943 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-6631f3dc-vg8sr"]
Oct 30 13:26:35.059239 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:26:35.059207    4943 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-6631f3dc-vg8sr"]
Oct 30 13:26:35.953975 ip-10-0-142-84 kubenswrapper[4943]: E1030 13:26:35.953941    4943 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-knqz6" containerName="extractor"
Oct 30 13:26:36.349252 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:26:36.349176    4943 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="92a51875-955c-40bd-994e-9ab0e8f46938" path="/var/lib/kubelet/pods/92a51875-955c-40bd-994e-9ab0e8f46938/volumes"
Oct 30 13:26:44.982835 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:26:44.982798    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-hdmrm"]
Oct 30 13:26:50.611278 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:26:50.611224    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-hdmrm"]
Oct 30 13:27:04.580249 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:27:04.580215    4943 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ajkfrv_17f5a8d2-67e0-4489-8734-00f0d4483788/extract/0.log"
Oct 30 13:27:04.590872 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:27:04.590843    4943 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ajkfrv_17f5a8d2-67e0-4489-8734-00f0d4483788/extract/0.log"
Oct 30 13:27:04.591596 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:27:04.591579    4943 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-samples-operator_cluster-samples-operator-6ddb6c8b94-7dtcd_23297e74-48f9-47b7-bb5a-80189bdbf2c4/cluster-samples-operator/0.log"
Oct 30 13:27:04.594126 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:27:04.594107    4943 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-8fqfr_d8930ee7-0ae0-48b5-8ce3-cb8820355cef/console-operator/2.log"
Oct 30 13:27:04.601878 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:27:04.601857    4943 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-samples-operator_cluster-samples-operator-6ddb6c8b94-7dtcd_23297e74-48f9-47b7-bb5a-80189bdbf2c4/cluster-samples-operator/0.log"
Oct 30 13:27:04.604424 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:27:04.604405    4943 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-8fqfr_d8930ee7-0ae0-48b5-8ce3-cb8820355cef/console-operator/2.log"
Oct 30 13:27:05.955025 ip-10-0-142-84 kubenswrapper[4943]: E1030 13:27:05.954986    4943 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-knqz6" containerName="extractor"
Oct 30 13:27:06.314573 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:27:06.314495    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-hdmrm"]
Oct 30 13:27:07.892204 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:27:07.892156    4943 scope.go:122] "RemoveContainer" containerID="60b9bfbc44521f71748b1ae34d58bd035f476f43db06394c522abb9d2bf937c0"
Oct 30 13:27:07.904247 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:27:07.904221    4943 scope.go:122] "RemoveContainer" containerID="d9af001b505d4c3b3668f2b8f6c768d161acdb871db59023a62fb8d8e0f20892"
Oct 30 13:27:07.912804 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:27:07.912780    4943 scope.go:122] "RemoveContainer" containerID="4d3ffa74191f55ca5667753b04d7d786f74eb1d290bba112f293fc4931fcfc12"
Oct 30 13:27:07.920770 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:27:07.920752    4943 scope.go:122] "RemoveContainer" containerID="93d2262feccf8b3f24095af2d9bb55ffbe292fc609c1ca7fb8d3310a77b08199"
Oct 30 13:27:07.929001 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:27:07.928987    4943 scope.go:122] "RemoveContainer" containerID="e1ddf29adc650c3523d96ebdb4d5b3450dea18f60f4cee4b5fcbca2e8a8b2253"
Oct 30 13:27:11.424920 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:27:11.424884    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-hdmrm"]
Oct 30 13:27:25.093068 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:27:25.093027    4943 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-85237e45-452wp"]
Oct 30 13:27:25.128178 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:27:25.128143    4943 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-85237e45-452wp"]
Oct 30 13:27:26.349734 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:27:26.349696    4943 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ec130f1c-eab5-4e6a-929c-b10d5f3d9283" path="/var/lib/kubelet/pods/ec130f1c-eab5-4e6a-929c-b10d5f3d9283/volumes"
Oct 30 13:27:34.001561 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:27:34.001524    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-hdmrm"]
Oct 30 13:27:35.957076 ip-10-0-142-84 kubenswrapper[4943]: E1030 13:27:35.957038    4943 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-knqz6" containerName="extractor"
Oct 30 13:27:38.830490 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:27:38.830453    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-hdmrm"]
Oct 30 13:27:59.577533 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:27:59.577311    4943 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-3eceab-65c875b454-7xqp8"]
Oct 30 13:27:59.579287 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:27:59.579232    4943 cpu_manager.go:414] "containerMap: removing container" podUID="0c0cc2dc-f621-4def-b952-174d35c4e244" containerName="revoke-keys"
Oct 30 13:27:59.579473 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:27:59.579457    4943 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0c0cc2dc-f621-4def-b952-174d35c4e244" containerName="revoke-keys"
Oct 30 13:27:59.579921 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:27:59.579900    4943 memory_manager.go:365] "RemoveStaleState removing state" podUID="0c0cc2dc-f621-4def-b952-174d35c4e244" containerName="revoke-keys"
Oct 30 13:27:59.585239 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:27:59.585209    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-3eceab-65c875b454-7xqp8"
Oct 30 13:27:59.607764 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:27:59.607728    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-3eceab-65c875b454-7xqp8"]
Oct 30 13:27:59.613920 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:27:59.613886    4943 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Oct 30 13:27:59.614114 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:27:59.613921    4943 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Oct 30 13:27:59.614114 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:27:59.613931    4943 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-5qrbm\""
Oct 30 13:27:59.614114 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:27:59.613997    4943 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-single-main-3eceab-serving-cert\""
Oct 30 13:27:59.614114 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:27:59.614001    4943 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Oct 30 13:27:59.694474 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:27:59.694438    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1daf6fd0-026e-434f-9592-0c1bc72a47fe-metrics-certs\") pod \"maas-api-e2e-worker-single-main-3eceab-65c875b454-7xqp8\" (UID: \"1daf6fd0-026e-434f-9592-0c1bc72a47fe\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-3eceab-65c875b454-7xqp8"
Oct 30 13:27:59.694666 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:27:59.694499    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t4bv7\" (UniqueName: \"kubernetes.io/projected/1daf6fd0-026e-434f-9592-0c1bc72a47fe-kube-api-access-t4bv7\") pod \"maas-api-e2e-worker-single-main-3eceab-65c875b454-7xqp8\" (UID: \"1daf6fd0-026e-434f-9592-0c1bc72a47fe\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-3eceab-65c875b454-7xqp8"
Oct 30 13:27:59.694666 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:27:59.694601    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1daf6fd0-026e-434f-9592-0c1bc72a47fe-openshift-service-ca\") pod \"maas-api-e2e-worker-single-main-3eceab-65c875b454-7xqp8\" (UID: \"1daf6fd0-026e-434f-9592-0c1bc72a47fe\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-3eceab-65c875b454-7xqp8"
Oct 30 13:27:59.694742 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:27:59.694717    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1daf6fd0-026e-434f-9592-0c1bc72a47fe-maas-api-tls\") pod \"maas-api-e2e-worker-single-main-3eceab-65c875b454-7xqp8\" (UID: \"1daf6fd0-026e-434f-9592-0c1bc72a47fe\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-3eceab-65c875b454-7xqp8"
Oct 30 13:27:59.795282 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:27:59.795223    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1daf6fd0-026e-434f-9592-0c1bc72a47fe-metrics-certs\") pod \"maas-api-e2e-worker-single-main-3eceab-65c875b454-7xqp8\" (UID: \"1daf6fd0-026e-434f-9592-0c1bc72a47fe\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-3eceab-65c875b454-7xqp8"
Oct 30 13:27:59.795502 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:27:59.795322    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t4bv7\" (UniqueName: \"kubernetes.io/projected/1daf6fd0-026e-434f-9592-0c1bc72a47fe-kube-api-access-t4bv7\") pod \"maas-api-e2e-worker-single-main-3eceab-65c875b454-7xqp8\" (UID: \"1daf6fd0-026e-434f-9592-0c1bc72a47fe\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-3eceab-65c875b454-7xqp8"
Oct 30 13:27:59.795502 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:27:59.795358    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1daf6fd0-026e-434f-9592-0c1bc72a47fe-openshift-service-ca\") pod \"maas-api-e2e-worker-single-main-3eceab-65c875b454-7xqp8\" (UID: \"1daf6fd0-026e-434f-9592-0c1bc72a47fe\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-3eceab-65c875b454-7xqp8"
Oct 30 13:27:59.795502 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:27:59.795433    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1daf6fd0-026e-434f-9592-0c1bc72a47fe-maas-api-tls\") pod \"maas-api-e2e-worker-single-main-3eceab-65c875b454-7xqp8\" (UID: \"1daf6fd0-026e-434f-9592-0c1bc72a47fe\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-3eceab-65c875b454-7xqp8"
Oct 30 13:27:59.796132 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:27:59.796102    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1daf6fd0-026e-434f-9592-0c1bc72a47fe-openshift-service-ca\") pod \"maas-api-e2e-worker-single-main-3eceab-65c875b454-7xqp8\" (UID: \"1daf6fd0-026e-434f-9592-0c1bc72a47fe\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-3eceab-65c875b454-7xqp8"
Oct 30 13:27:59.797899 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:27:59.797867    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1daf6fd0-026e-434f-9592-0c1bc72a47fe-maas-api-tls\") pod \"maas-api-e2e-worker-single-main-3eceab-65c875b454-7xqp8\" (UID: \"1daf6fd0-026e-434f-9592-0c1bc72a47fe\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-3eceab-65c875b454-7xqp8"
Oct 30 13:27:59.798054 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:27:59.797914    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1daf6fd0-026e-434f-9592-0c1bc72a47fe-metrics-certs\") pod \"maas-api-e2e-worker-single-main-3eceab-65c875b454-7xqp8\" (UID: \"1daf6fd0-026e-434f-9592-0c1bc72a47fe\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-3eceab-65c875b454-7xqp8"
Oct 30 13:27:59.823698 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:27:59.823664    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t4bv7\" (UniqueName: \"kubernetes.io/projected/1daf6fd0-026e-434f-9592-0c1bc72a47fe-kube-api-access-t4bv7\") pod \"maas-api-e2e-worker-single-main-3eceab-65c875b454-7xqp8\" (UID: \"1daf6fd0-026e-434f-9592-0c1bc72a47fe\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-3eceab-65c875b454-7xqp8"
Oct 30 13:27:59.907776 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:27:59.907680    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-3eceab-65c875b454-7xqp8"
Oct 30 13:28:00.215411 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:28:00.215352    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-3eceab-65c875b454-7xqp8"]
Oct 30 13:28:00.215734 ip-10-0-142-84 kubenswrapper[4943]: W1030 13:28:00.215706    4943 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1daf6fd0_026e_434f_9592_0c1bc72a47fe.slice/crio-0f998c6da8f65b12ceae813231dff75d35de733e310e958e7ff6e0edef1e4f9b WatchSource:0}: Error finding container 0f998c6da8f65b12ceae813231dff75d35de733e310e958e7ff6e0edef1e4f9b: Status 404 returned error can't find the container with id 0f998c6da8f65b12ceae813231dff75d35de733e310e958e7ff6e0edef1e4f9b
Oct 30 13:28:00.217203 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:28:00.217185    4943 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 13:28:01.122228 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:28:01.122190    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-3eceab-65c875b454-7xqp8" event={"ID":"1daf6fd0-026e-434f-9592-0c1bc72a47fe","Type":"ContainerStarted","Data":"d9c12bbe2452a023d7fbae6a29385bf1a7b99f61a82d6ae0edf959cd35a7d29b"}
Oct 30 13:28:01.122747 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:28:01.122243    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-3eceab-65c875b454-7xqp8" event={"ID":"1daf6fd0-026e-434f-9592-0c1bc72a47fe","Type":"ContainerStarted","Data":"0f998c6da8f65b12ceae813231dff75d35de733e310e958e7ff6e0edef1e4f9b"}
Oct 30 13:28:01.122747 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:28:01.122321    4943 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-3eceab-65c875b454-7xqp8"
Oct 30 13:28:01.168952 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:28:01.168899    4943 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-3eceab-65c875b454-7xqp8" podStartSLOduration=1.7616990829999999 podStartE2EDuration="2.168884896s" totalImagesPullingTime="407.185813ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:27:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:28:01.168746327 +0000 UTC m=+1856.956750058" watchObservedRunningTime="2026-10-30 13:28:01.168884896 +0000 UTC m=+1856.956888627"
Oct 30 13:28:03.398282 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:28:03.398172    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-hdmrm"]
Oct 30 13:28:05.955093 ip-10-0-142-84 kubenswrapper[4943]: E1030 13:28:05.955054    4943 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-knqz6" containerName="extractor"
Oct 30 13:28:07.135166 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:28:07.135131    4943 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-3eceab-65c875b454-7xqp8"
Oct 30 13:28:07.967380 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:28:07.967340    4943 scope.go:122] "RemoveContainer" containerID="9053a006ca5e71bbf11cd6aaa0c2198fd0bcf4b2ae2f2f356e3835f32c1bd754"
Oct 30 13:28:07.977324 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:28:07.977296    4943 scope.go:122] "RemoveContainer" containerID="2acce7af2aba9816cfc70604ce6451017e0fca1506f5cdb6c9e2ccdbfb469c5f"
Oct 30 13:28:07.987173 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:28:07.987147    4943 scope.go:122] "RemoveContainer" containerID="e2a612caa8cab1756c976f5ec2da9e734210935ae608a8342fc803db964c2c7f"
Oct 30 13:28:28.060853 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:28:28.060810    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-hdmrm"]
Oct 30 13:28:35.956136 ip-10-0-142-84 kubenswrapper[4943]: E1030 13:28:35.956095    4943 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-knqz6" containerName="extractor"
Oct 30 13:28:40.037371 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:28:40.037335    4943 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-c1e271a9-g4m2m"]
Oct 30 13:28:40.050901 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:28:40.050868    4943 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-c1e271a9-g4m2m"]
Oct 30 13:28:40.349821 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:28:40.349739    4943 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="143e6e24-96e1-47bb-ba3a-1aa21b7bc699" path="/var/lib/kubelet/pods/143e6e24-96e1-47bb-ba3a-1aa21b7bc699/volumes"
Oct 30 13:28:42.922191 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:28:42.922154    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-hdmrm"]
Oct 30 13:29:00.034995 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:29:00.034954    4943 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-2e1f5b5f-j5bfx"]
Oct 30 13:29:00.048809 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:29:00.048771    4943 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-2e1f5b5f-j5bfx"]
Oct 30 13:29:00.349859 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:29:00.349817    4943 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b84d7af2-77e6-4d59-844c-e8a9ab7279f8" path="/var/lib/kubelet/pods/b84d7af2-77e6-4d59-844c-e8a9ab7279f8/volumes"
Oct 30 13:29:00.399881 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:29:00.399846    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-hdmrm"]
Oct 30 13:29:05.955558 ip-10-0-142-84 kubenswrapper[4943]: E1030 13:29:05.955518    4943 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-knqz6" containerName="extractor"
Oct 30 13:29:08.022703 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:29:08.022667    4943 scope.go:122] "RemoveContainer" containerID="99f14419fcbd92fe06ba0113c75001df18a562b180cc336d217ea1dbf34c3c81"
Oct 30 13:29:11.035348 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:29:11.035295    4943 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-ec843018-6qx99"]
Oct 30 13:29:11.049632 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:29:11.049591    4943 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-ec843018-6qx99"]
Oct 30 13:29:11.996750 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:29:11.996710    4943 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-3eceab-65c875b454-7xqp8"]
Oct 30 13:29:11.997029 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:29:11.996985    4943 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-3eceab-65c875b454-7xqp8" podUID="1daf6fd0-026e-434f-9592-0c1bc72a47fe" containerName="maas-api" containerID="cri-o://d9c12bbe2452a023d7fbae6a29385bf1a7b99f61a82d6ae0edf959cd35a7d29b" gracePeriod=30
Oct 30 13:29:12.130225 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:29:12.130175    4943 prober.go:122] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-3eceab-65c875b454-7xqp8" podUID="1daf6fd0-026e-434f-9592-0c1bc72a47fe" containerName="maas-api" probeResult="failure" output="Get \"https://10.132.0.142:8443/health\": dial tcp 10.132.0.142:8443: connect: connection refused"
Oct 30 13:29:12.350369 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:29:12.350336    4943 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f7b09437-8214-40b9-a2be-ad764125bd09" path="/var/lib/kubelet/pods/f7b09437-8214-40b9-a2be-ad764125bd09/volumes"
Oct 30 13:29:12.458990 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:29:12.458962    4943 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-3eceab-65c875b454-7xqp8"
Oct 30 13:29:12.549460 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:29:12.549360    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/1daf6fd0-026e-434f-9592-0c1bc72a47fe-metrics-certs\" (UniqueName: \"kubernetes.io/secret/1daf6fd0-026e-434f-9592-0c1bc72a47fe-metrics-certs\") pod \"1daf6fd0-026e-434f-9592-0c1bc72a47fe\" (UID: \"1daf6fd0-026e-434f-9592-0c1bc72a47fe\") "
Oct 30 13:29:12.549460 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:29:12.549424    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1daf6fd0-026e-434f-9592-0c1bc72a47fe-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1daf6fd0-026e-434f-9592-0c1bc72a47fe-openshift-service-ca\") pod \"1daf6fd0-026e-434f-9592-0c1bc72a47fe\" (UID: \"1daf6fd0-026e-434f-9592-0c1bc72a47fe\") "
Oct 30 13:29:12.549705 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:29:12.549510    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1daf6fd0-026e-434f-9592-0c1bc72a47fe-kube-api-access-t4bv7\" (UniqueName: \"kubernetes.io/projected/1daf6fd0-026e-434f-9592-0c1bc72a47fe-kube-api-access-t4bv7\") pod \"1daf6fd0-026e-434f-9592-0c1bc72a47fe\" (UID: \"1daf6fd0-026e-434f-9592-0c1bc72a47fe\") "
Oct 30 13:29:12.549705 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:29:12.549567    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/1daf6fd0-026e-434f-9592-0c1bc72a47fe-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1daf6fd0-026e-434f-9592-0c1bc72a47fe-maas-api-tls\") pod \"1daf6fd0-026e-434f-9592-0c1bc72a47fe\" (UID: \"1daf6fd0-026e-434f-9592-0c1bc72a47fe\") "
Oct 30 13:29:12.549874 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:29:12.549844    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1daf6fd0-026e-434f-9592-0c1bc72a47fe-openshift-service-ca" pod "1daf6fd0-026e-434f-9592-0c1bc72a47fe" (UID: "1daf6fd0-026e-434f-9592-0c1bc72a47fe"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 13:29:12.551715 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:29:12.551649    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1daf6fd0-026e-434f-9592-0c1bc72a47fe-maas-api-tls" pod "1daf6fd0-026e-434f-9592-0c1bc72a47fe" (UID: "1daf6fd0-026e-434f-9592-0c1bc72a47fe"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 13:29:12.551715 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:29:12.551702    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1daf6fd0-026e-434f-9592-0c1bc72a47fe-metrics-certs" pod "1daf6fd0-026e-434f-9592-0c1bc72a47fe" (UID: "1daf6fd0-026e-434f-9592-0c1bc72a47fe"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 13:29:12.551891 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:29:12.551720    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1daf6fd0-026e-434f-9592-0c1bc72a47fe-kube-api-access-t4bv7" pod "1daf6fd0-026e-434f-9592-0c1bc72a47fe" (UID: "1daf6fd0-026e-434f-9592-0c1bc72a47fe"). InnerVolumeSpecName "kube-api-access-t4bv7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:29:12.565501 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:29:12.565469    4943 generic.go:356] "Generic (PLEG): container finished" podID="1daf6fd0-026e-434f-9592-0c1bc72a47fe" containerID="d9c12bbe2452a023d7fbae6a29385bf1a7b99f61a82d6ae0edf959cd35a7d29b" exitCode=0
Oct 30 13:29:12.565638 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:29:12.565518    4943 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-3eceab-65c875b454-7xqp8"
Oct 30 13:29:12.565638 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:29:12.565544    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-3eceab-65c875b454-7xqp8" event={"ID":"1daf6fd0-026e-434f-9592-0c1bc72a47fe","Type":"ContainerDied","Data":"d9c12bbe2452a023d7fbae6a29385bf1a7b99f61a82d6ae0edf959cd35a7d29b"}
Oct 30 13:29:12.565638 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:29:12.565605    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-3eceab-65c875b454-7xqp8" event={"ID":"1daf6fd0-026e-434f-9592-0c1bc72a47fe","Type":"ContainerDied","Data":"0f998c6da8f65b12ceae813231dff75d35de733e310e958e7ff6e0edef1e4f9b"}
Oct 30 13:29:12.565638 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:29:12.565629    4943 scope.go:122] "RemoveContainer" containerID="d9c12bbe2452a023d7fbae6a29385bf1a7b99f61a82d6ae0edf959cd35a7d29b"
Oct 30 13:29:12.578445 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:29:12.578427    4943 scope.go:122] "RemoveContainer" containerID="d9c12bbe2452a023d7fbae6a29385bf1a7b99f61a82d6ae0edf959cd35a7d29b"
Oct 30 13:29:12.578736 ip-10-0-142-84 kubenswrapper[4943]: E1030 13:29:12.578715    4943 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d9c12bbe2452a023d7fbae6a29385bf1a7b99f61a82d6ae0edf959cd35a7d29b\": container with ID starting with d9c12bbe2452a023d7fbae6a29385bf1a7b99f61a82d6ae0edf959cd35a7d29b not found: ID does not exist" containerID="d9c12bbe2452a023d7fbae6a29385bf1a7b99f61a82d6ae0edf959cd35a7d29b"
Oct 30 13:29:12.578794 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:29:12.578745    4943 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d9c12bbe2452a023d7fbae6a29385bf1a7b99f61a82d6ae0edf959cd35a7d29b"} err="failed to get container status \"d9c12bbe2452a023d7fbae6a29385bf1a7b99f61a82d6ae0edf959cd35a7d29b\": rpc error: code = NotFound desc = could not find container \"d9c12bbe2452a023d7fbae6a29385bf1a7b99f61a82d6ae0edf959cd35a7d29b\": container with ID starting with d9c12bbe2452a023d7fbae6a29385bf1a7b99f61a82d6ae0edf959cd35a7d29b not found: ID does not exist"
Oct 30 13:29:12.625402 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:29:12.625361    4943 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-3eceab-65c875b454-7xqp8"]
Oct 30 13:29:12.627806 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:29:12.627774    4943 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-3eceab-65c875b454-7xqp8"]
Oct 30 13:29:12.650660 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:29:12.650620    4943 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1daf6fd0-026e-434f-9592-0c1bc72a47fe-openshift-service-ca\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:29:12.650660 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:29:12.650653    4943 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t4bv7\" (UniqueName: \"kubernetes.io/projected/1daf6fd0-026e-434f-9592-0c1bc72a47fe-kube-api-access-t4bv7\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:29:12.650868 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:29:12.650669    4943 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1daf6fd0-026e-434f-9592-0c1bc72a47fe-maas-api-tls\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:29:12.650868 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:29:12.650681    4943 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1daf6fd0-026e-434f-9592-0c1bc72a47fe-metrics-certs\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:29:14.349544 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:29:14.349502    4943 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1daf6fd0-026e-434f-9592-0c1bc72a47fe" path="/var/lib/kubelet/pods/1daf6fd0-026e-434f-9592-0c1bc72a47fe/volumes"
Oct 30 13:29:14.434285 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:29:14.430630    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-hdmrm"]
Oct 30 13:29:22.427133 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:29:22.427052    4943 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-87wwj"]
Oct 30 13:29:22.427544 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:29:22.427511    4943 cpu_manager.go:414] "containerMap: removing container" podUID="1daf6fd0-026e-434f-9592-0c1bc72a47fe" containerName="maas-api"
Oct 30 13:29:22.427544 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:29:22.427523    4943 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1daf6fd0-026e-434f-9592-0c1bc72a47fe" containerName="maas-api"
Oct 30 13:29:22.427617 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:29:22.427598    4943 memory_manager.go:365] "RemoveStaleState removing state" podUID="1daf6fd0-026e-434f-9592-0c1bc72a47fe" containerName="maas-api"
Oct 30 13:29:22.434412 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:29:22.434377    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-87wwj"
Oct 30 13:29:22.451564 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:29:22.451529    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-87wwj"]
Oct 30 13:29:22.462256 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:29:22.462220    4943 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 13:29:22.462530 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:29:22.462484    4943 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Oct 30 13:29:22.462663 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:29:22.462554    4943 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Oct 30 13:29:22.466434 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:29:22.466413    4943 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-p9z57\""
Oct 30 13:29:22.468005 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:29:22.467987    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/4207d02f-ec71-4f06-9d6b-75c7e43a45f8-maas-api-xks-ca\") pod \"maas-api-revoke-sub-3d4156f1-87wwj\" (UID: \"4207d02f-ec71-4f06-9d6b-75c7e43a45f8\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-87wwj"
Oct 30 13:29:22.468067 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:29:22.468019    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-klxpv\" (UniqueName: \"kubernetes.io/projected/4207d02f-ec71-4f06-9d6b-75c7e43a45f8-kube-api-access-klxpv\") pod \"maas-api-revoke-sub-3d4156f1-87wwj\" (UID: \"4207d02f-ec71-4f06-9d6b-75c7e43a45f8\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-87wwj"
Oct 30 13:29:22.468145 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:29:22.468128    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/4207d02f-ec71-4f06-9d6b-75c7e43a45f8-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-3d4156f1-87wwj\" (UID: \"4207d02f-ec71-4f06-9d6b-75c7e43a45f8\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-87wwj"
Oct 30 13:29:22.569056 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:29:22.569016    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/4207d02f-ec71-4f06-9d6b-75c7e43a45f8-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-3d4156f1-87wwj\" (UID: \"4207d02f-ec71-4f06-9d6b-75c7e43a45f8\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-87wwj"
Oct 30 13:29:22.569236 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:29:22.569131    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/4207d02f-ec71-4f06-9d6b-75c7e43a45f8-maas-api-xks-ca\") pod \"maas-api-revoke-sub-3d4156f1-87wwj\" (UID: \"4207d02f-ec71-4f06-9d6b-75c7e43a45f8\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-87wwj"
Oct 30 13:29:22.569236 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:29:22.569159    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-klxpv\" (UniqueName: \"kubernetes.io/projected/4207d02f-ec71-4f06-9d6b-75c7e43a45f8-kube-api-access-klxpv\") pod \"maas-api-revoke-sub-3d4156f1-87wwj\" (UID: \"4207d02f-ec71-4f06-9d6b-75c7e43a45f8\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-87wwj"
Oct 30 13:29:22.569758 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:29:22.569736    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/4207d02f-ec71-4f06-9d6b-75c7e43a45f8-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-3d4156f1-87wwj\" (UID: \"4207d02f-ec71-4f06-9d6b-75c7e43a45f8\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-87wwj"
Oct 30 13:29:22.571638 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:29:22.571616    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/4207d02f-ec71-4f06-9d6b-75c7e43a45f8-maas-api-xks-ca\") pod \"maas-api-revoke-sub-3d4156f1-87wwj\" (UID: \"4207d02f-ec71-4f06-9d6b-75c7e43a45f8\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-87wwj"
Oct 30 13:29:22.595293 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:29:22.595243    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-klxpv\" (UniqueName: \"kubernetes.io/projected/4207d02f-ec71-4f06-9d6b-75c7e43a45f8-kube-api-access-klxpv\") pod \"maas-api-revoke-sub-3d4156f1-87wwj\" (UID: \"4207d02f-ec71-4f06-9d6b-75c7e43a45f8\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-87wwj"
Oct 30 13:29:22.750583 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:29:22.750496    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-87wwj"
Oct 30 13:29:23.061454 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:29:23.061420    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-87wwj"]
Oct 30 13:29:23.062667 ip-10-0-142-84 kubenswrapper[4943]: W1030 13:29:23.062640    4943 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4207d02f_ec71_4f06_9d6b_75c7e43a45f8.slice/crio-27fb5cee7dcf0062563462edb0c7cc22af714d223f245bb5fa412c84380777e9 WatchSource:0}: Error finding container 27fb5cee7dcf0062563462edb0c7cc22af714d223f245bb5fa412c84380777e9: Status 404 returned error can't find the container with id 27fb5cee7dcf0062563462edb0c7cc22af714d223f245bb5fa412c84380777e9
Oct 30 13:29:23.631347 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:29:23.631235    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-87wwj" event={"ID":"4207d02f-ec71-4f06-9d6b-75c7e43a45f8","Type":"ContainerStarted","Data":"ba2ab005a3f9298354e558dc5f7d9ec9bbc55d96d3e9464a957c236c6aa77493"}
Oct 30 13:29:23.631347 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:29:23.631318    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-87wwj" event={"ID":"4207d02f-ec71-4f06-9d6b-75c7e43a45f8","Type":"ContainerStarted","Data":"27fb5cee7dcf0062563462edb0c7cc22af714d223f245bb5fa412c84380777e9"}
Oct 30 13:29:23.681204 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:29:23.681146    4943 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-87wwj" podStartSLOduration=1.681131532 podStartE2EDuration="1.681131532s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:29:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:29:23.679041548 +0000 UTC m=+1939.467045277" watchObservedRunningTime="2026-10-30 13:29:23.681131532 +0000 UTC m=+1939.469135261"
Oct 30 13:29:24.640458 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:29:24.640416    4943 generic.go:356] "Generic (PLEG): container finished" podID="4207d02f-ec71-4f06-9d6b-75c7e43a45f8" containerID="ba2ab005a3f9298354e558dc5f7d9ec9bbc55d96d3e9464a957c236c6aa77493" exitCode=0
Oct 30 13:29:24.641032 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:29:24.640471    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-87wwj" event={"ID":"4207d02f-ec71-4f06-9d6b-75c7e43a45f8","Type":"ContainerDied","Data":"ba2ab005a3f9298354e558dc5f7d9ec9bbc55d96d3e9464a957c236c6aa77493"}
Oct 30 13:29:25.795745 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:29:25.795718    4943 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-87wwj"
Oct 30 13:29:25.892396 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:29:25.892360    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/4207d02f-ec71-4f06-9d6b-75c7e43a45f8-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/4207d02f-ec71-4f06-9d6b-75c7e43a45f8-maas-api-openshift-ca\") pod \"4207d02f-ec71-4f06-9d6b-75c7e43a45f8\" (UID: \"4207d02f-ec71-4f06-9d6b-75c7e43a45f8\") "
Oct 30 13:29:25.892612 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:29:25.892428    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/4207d02f-ec71-4f06-9d6b-75c7e43a45f8-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/4207d02f-ec71-4f06-9d6b-75c7e43a45f8-maas-api-xks-ca\") pod \"4207d02f-ec71-4f06-9d6b-75c7e43a45f8\" (UID: \"4207d02f-ec71-4f06-9d6b-75c7e43a45f8\") "
Oct 30 13:29:25.892612 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:29:25.892518    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4207d02f-ec71-4f06-9d6b-75c7e43a45f8-kube-api-access-klxpv\" (UniqueName: \"kubernetes.io/projected/4207d02f-ec71-4f06-9d6b-75c7e43a45f8-kube-api-access-klxpv\") pod \"4207d02f-ec71-4f06-9d6b-75c7e43a45f8\" (UID: \"4207d02f-ec71-4f06-9d6b-75c7e43a45f8\") "
Oct 30 13:29:25.892786 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:29:25.892760    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4207d02f-ec71-4f06-9d6b-75c7e43a45f8-maas-api-openshift-ca" pod "4207d02f-ec71-4f06-9d6b-75c7e43a45f8" (UID: "4207d02f-ec71-4f06-9d6b-75c7e43a45f8"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 13:29:25.894617 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:29:25.894592    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4207d02f-ec71-4f06-9d6b-75c7e43a45f8-maas-api-xks-ca" pod "4207d02f-ec71-4f06-9d6b-75c7e43a45f8" (UID: "4207d02f-ec71-4f06-9d6b-75c7e43a45f8"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 13:29:25.894617 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:29:25.894606    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4207d02f-ec71-4f06-9d6b-75c7e43a45f8-kube-api-access-klxpv" pod "4207d02f-ec71-4f06-9d6b-75c7e43a45f8" (UID: "4207d02f-ec71-4f06-9d6b-75c7e43a45f8"). InnerVolumeSpecName "kube-api-access-klxpv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:29:25.993698 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:29:25.993655    4943 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/4207d02f-ec71-4f06-9d6b-75c7e43a45f8-maas-api-xks-ca\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:29:25.993698 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:29:25.993689    4943 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-klxpv\" (UniqueName: \"kubernetes.io/projected/4207d02f-ec71-4f06-9d6b-75c7e43a45f8-kube-api-access-klxpv\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:29:25.993698 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:29:25.993703    4943 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/4207d02f-ec71-4f06-9d6b-75c7e43a45f8-maas-api-openshift-ca\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:29:26.653532 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:29:26.653439    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-87wwj" event={"ID":"4207d02f-ec71-4f06-9d6b-75c7e43a45f8","Type":"ContainerDied","Data":"27fb5cee7dcf0062563462edb0c7cc22af714d223f245bb5fa412c84380777e9"}
Oct 30 13:29:26.653532 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:29:26.653461    4943 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-87wwj"
Oct 30 13:29:26.653532 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:29:26.653495    4943 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="27fb5cee7dcf0062563462edb0c7cc22af714d223f245bb5fa412c84380777e9"
Oct 30 13:29:34.424198 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:29:34.424152    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-hdmrm"]
Oct 30 13:29:36.101881 ip-10-0-142-84 kubenswrapper[4943]: E1030 13:29:36.101835    4943 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-knqz6" containerName="extractor"
Oct 30 13:29:39.230741 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:29:39.230705    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-hdmrm"]
Oct 30 13:29:49.039415 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:29:49.039374    4943 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-046e1343-g8p7z"]
Oct 30 13:29:49.053525 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:29:49.053483    4943 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-046e1343-g8p7z"]
Oct 30 13:29:50.349717 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:29:50.349682    4943 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="29e6fafd-e405-4c80-bde1-3c76ae91e50f" path="/var/lib/kubelet/pods/29e6fafd-e405-4c80-bde1-3c76ae91e50f/volumes"
Oct 30 13:30:00.034900 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:30:00.034856    4943 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-d3fba056-j59bv"]
Oct 30 13:30:00.048767 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:30:00.048733    4943 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-d3fba056-j59bv"]
Oct 30 13:30:00.349756 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:30:00.349721    4943 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f50b6e2c-85e9-49b5-87e2-83377054f324" path="/var/lib/kubelet/pods/f50b6e2c-85e9-49b5-87e2-83377054f324/volumes"
Oct 30 13:30:05.955678 ip-10-0-142-84 kubenswrapper[4943]: E1030 13:30:05.955637    4943 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-knqz6" containerName="extractor"
Oct 30 13:30:08.049025 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:30:08.048990    4943 scope.go:122] "RemoveContainer" containerID="d5cb7effea101c94b4ee77e35f11234b54ddf8a2768619f121bac8476da1a522"
Oct 30 13:30:08.058413 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:30:08.058387    4943 scope.go:122] "RemoveContainer" containerID="7ca1e6087058579c298fc9d7ffb917e2e3e2973732b161626f2ef324e0cb09d3"
Oct 30 13:30:26.043762 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:30:26.043723    4943 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-ae594979-wt4d9"]
Oct 30 13:30:26.057528 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:30:26.057493    4943 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-ae594979-wt4d9"]
Oct 30 13:30:26.350222 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:30:26.350142    4943 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0c0cc2dc-f621-4def-b952-174d35c4e244" path="/var/lib/kubelet/pods/0c0cc2dc-f621-4def-b952-174d35c4e244/volumes"
Oct 30 13:30:35.954566 ip-10-0-142-84 kubenswrapper[4943]: E1030 13:30:35.954527    4943 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-knqz6" containerName="extractor"
Oct 30 13:30:42.876694 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:30:42.876649    4943 prober.go:122] "Probe failed" probeType="Liveness" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-84.ec2.internal" podUID="0c9cf3ec0e3ab50517e6f80b07fb5fb2" containerName="haproxy" probeResult="failure" output="Get \"https://172.20.0.1:6443/version\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:31:05.954327 ip-10-0-142-84 kubenswrapper[4943]: E1030 13:31:05.954288    4943 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-knqz6" containerName="extractor"
Oct 30 13:31:08.089506 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:31:08.089476    4943 scope.go:122] "RemoveContainer" containerID="dfd227df1403c1380d7bf706f409bd110d6aa1d6178c21a88898de6f6b0fddca"
Oct 30 13:31:08.099082 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:31:08.099057    4943 scope.go:122] "RemoveContainer" containerID="b2d6b337254c7cf43711f3ff3680686fb6fa0cfbf37be56e52491f0be86fa161"
Oct 30 13:31:08.107306 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:31:08.107287    4943 scope.go:122] "RemoveContainer" containerID="ae735397781c16b3b664b9b6353de739c3d4ec1dfb93097e09911dcd3f69b30b"
Oct 30 13:31:18.888103 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:31:18.888068    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-hdmrm"]
Oct 30 13:31:33.926116 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:31:33.926080    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-hdmrm"]
Oct 30 13:31:35.955313 ip-10-0-142-84 kubenswrapper[4943]: E1030 13:31:35.955275    4943 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-knqz6" containerName="extractor"
Oct 30 13:32:04.621613 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:32:04.621584    4943 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ajkfrv_17f5a8d2-67e0-4489-8734-00f0d4483788/extract/0.log"
Oct 30 13:32:04.632763 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:32:04.632735    4943 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-samples-operator_cluster-samples-operator-6ddb6c8b94-7dtcd_23297e74-48f9-47b7-bb5a-80189bdbf2c4/cluster-samples-operator/0.log"
Oct 30 13:32:04.635001 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:32:04.634981    4943 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-8fqfr_d8930ee7-0ae0-48b5-8ce3-cb8820355cef/console-operator/2.log"
Oct 30 13:32:04.635396 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:32:04.635374    4943 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ajkfrv_17f5a8d2-67e0-4489-8734-00f0d4483788/extract/0.log"
Oct 30 13:32:04.645240 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:32:04.645221    4943 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-samples-operator_cluster-samples-operator-6ddb6c8b94-7dtcd_23297e74-48f9-47b7-bb5a-80189bdbf2c4/cluster-samples-operator/0.log"
Oct 30 13:32:04.647336 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:32:04.647317    4943 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-8fqfr_d8930ee7-0ae0-48b5-8ce3-cb8820355cef/console-operator/2.log"
Oct 30 13:32:05.953246 ip-10-0-142-84 kubenswrapper[4943]: E1030 13:32:05.953209    4943 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-knqz6" containerName="extractor"
Oct 30 13:32:07.838248 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:32:07.838210    4943 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-c78854-59f4c49558-vfdh9"]
Oct 30 13:32:07.838707 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:32:07.838692    4943 cpu_manager.go:414] "containerMap: removing container" podUID="4207d02f-ec71-4f06-9d6b-75c7e43a45f8" containerName="revoke-keys"
Oct 30 13:32:07.838751 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:32:07.838708    4943 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4207d02f-ec71-4f06-9d6b-75c7e43a45f8" containerName="revoke-keys"
Oct 30 13:32:07.838808 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:32:07.838799    4943 memory_manager.go:365] "RemoveStaleState removing state" podUID="4207d02f-ec71-4f06-9d6b-75c7e43a45f8" containerName="revoke-keys"
Oct 30 13:32:07.844701 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:32:07.844675    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-c78854-59f4c49558-vfdh9"
Oct 30 13:32:07.875412 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:32:07.875287    4943 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Oct 30 13:32:07.875412 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:32:07.875383    4943 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Oct 30 13:32:07.882182 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:32:07.882154    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-c78854-59f4c49558-vfdh9"]
Oct 30 13:32:07.908112 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:32:07.908083    4943 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-single-main-c78854-serving-cert\""
Oct 30 13:32:07.908568 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:32:07.908549    4943 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-5qrbm\""
Oct 30 13:32:07.911407 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:32:07.911387    4943 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Oct 30 13:32:07.992550 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:32:07.992503    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6p5gh\" (UniqueName: \"kubernetes.io/projected/0b7cc484-1df0-4838-8bac-ad5229817bf8-kube-api-access-6p5gh\") pod \"maas-api-e2e-worker-single-main-c78854-59f4c49558-vfdh9\" (UID: \"0b7cc484-1df0-4838-8bac-ad5229817bf8\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-c78854-59f4c49558-vfdh9"
Oct 30 13:32:07.992550 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:32:07.992555    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0b7cc484-1df0-4838-8bac-ad5229817bf8-metrics-certs\") pod \"maas-api-e2e-worker-single-main-c78854-59f4c49558-vfdh9\" (UID: \"0b7cc484-1df0-4838-8bac-ad5229817bf8\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-c78854-59f4c49558-vfdh9"
Oct 30 13:32:07.992745 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:32:07.992708    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0b7cc484-1df0-4838-8bac-ad5229817bf8-maas-api-tls\") pod \"maas-api-e2e-worker-single-main-c78854-59f4c49558-vfdh9\" (UID: \"0b7cc484-1df0-4838-8bac-ad5229817bf8\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-c78854-59f4c49558-vfdh9"
Oct 30 13:32:07.992816 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:32:07.992799    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0b7cc484-1df0-4838-8bac-ad5229817bf8-openshift-service-ca\") pod \"maas-api-e2e-worker-single-main-c78854-59f4c49558-vfdh9\" (UID: \"0b7cc484-1df0-4838-8bac-ad5229817bf8\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-c78854-59f4c49558-vfdh9"
Oct 30 13:32:08.094444 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:32:08.094353    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6p5gh\" (UniqueName: \"kubernetes.io/projected/0b7cc484-1df0-4838-8bac-ad5229817bf8-kube-api-access-6p5gh\") pod \"maas-api-e2e-worker-single-main-c78854-59f4c49558-vfdh9\" (UID: \"0b7cc484-1df0-4838-8bac-ad5229817bf8\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-c78854-59f4c49558-vfdh9"
Oct 30 13:32:08.094444 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:32:08.094397    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0b7cc484-1df0-4838-8bac-ad5229817bf8-metrics-certs\") pod \"maas-api-e2e-worker-single-main-c78854-59f4c49558-vfdh9\" (UID: \"0b7cc484-1df0-4838-8bac-ad5229817bf8\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-c78854-59f4c49558-vfdh9"
Oct 30 13:32:08.094672 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:32:08.094451    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0b7cc484-1df0-4838-8bac-ad5229817bf8-maas-api-tls\") pod \"maas-api-e2e-worker-single-main-c78854-59f4c49558-vfdh9\" (UID: \"0b7cc484-1df0-4838-8bac-ad5229817bf8\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-c78854-59f4c49558-vfdh9"
Oct 30 13:32:08.094672 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:32:08.094512    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0b7cc484-1df0-4838-8bac-ad5229817bf8-openshift-service-ca\") pod \"maas-api-e2e-worker-single-main-c78854-59f4c49558-vfdh9\" (UID: \"0b7cc484-1df0-4838-8bac-ad5229817bf8\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-c78854-59f4c49558-vfdh9"
Oct 30 13:32:08.094672 ip-10-0-142-84 kubenswrapper[4943]: E1030 13:32:08.094643    4943 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-worker-single-main-c78854-serving-cert: secret "maas-api-e2e-worker-single-main-c78854-serving-cert" not found
Oct 30 13:32:08.094785 ip-10-0-142-84 kubenswrapper[4943]: E1030 13:32:08.094738    4943 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0b7cc484-1df0-4838-8bac-ad5229817bf8-maas-api-tls podName:0b7cc484-1df0-4838-8bac-ad5229817bf8 nodeName:}" failed. No retries permitted until 2026-10-30 13:32:08.594713191 +0000 UTC m=+2104.382716903 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/0b7cc484-1df0-4838-8bac-ad5229817bf8-maas-api-tls") pod "maas-api-e2e-worker-single-main-c78854-59f4c49558-vfdh9" (UID: "0b7cc484-1df0-4838-8bac-ad5229817bf8") : secret "maas-api-e2e-worker-single-main-c78854-serving-cert" not found
Oct 30 13:32:08.095276 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:32:08.095246    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0b7cc484-1df0-4838-8bac-ad5229817bf8-openshift-service-ca\") pod \"maas-api-e2e-worker-single-main-c78854-59f4c49558-vfdh9\" (UID: \"0b7cc484-1df0-4838-8bac-ad5229817bf8\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-c78854-59f4c49558-vfdh9"
Oct 30 13:32:08.097003 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:32:08.096981    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0b7cc484-1df0-4838-8bac-ad5229817bf8-metrics-certs\") pod \"maas-api-e2e-worker-single-main-c78854-59f4c49558-vfdh9\" (UID: \"0b7cc484-1df0-4838-8bac-ad5229817bf8\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-c78854-59f4c49558-vfdh9"
Oct 30 13:32:08.126072 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:32:08.126040    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6p5gh\" (UniqueName: \"kubernetes.io/projected/0b7cc484-1df0-4838-8bac-ad5229817bf8-kube-api-access-6p5gh\") pod \"maas-api-e2e-worker-single-main-c78854-59f4c49558-vfdh9\" (UID: \"0b7cc484-1df0-4838-8bac-ad5229817bf8\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-c78854-59f4c49558-vfdh9"
Oct 30 13:32:08.137441 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:32:08.137418    4943 scope.go:122] "RemoveContainer" containerID="d283650d4ca806731bd6a64abebcb33477b9fc6d91bfb7f61615092d6676f388"
Oct 30 13:32:08.603546 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:32:08.603504    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0b7cc484-1df0-4838-8bac-ad5229817bf8-maas-api-tls\") pod \"maas-api-e2e-worker-single-main-c78854-59f4c49558-vfdh9\" (UID: \"0b7cc484-1df0-4838-8bac-ad5229817bf8\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-c78854-59f4c49558-vfdh9"
Oct 30 13:32:08.605979 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:32:08.605946    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0b7cc484-1df0-4838-8bac-ad5229817bf8-maas-api-tls\") pod \"maas-api-e2e-worker-single-main-c78854-59f4c49558-vfdh9\" (UID: \"0b7cc484-1df0-4838-8bac-ad5229817bf8\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-c78854-59f4c49558-vfdh9"
Oct 30 13:32:08.758638 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:32:08.758605    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-c78854-59f4c49558-vfdh9"
Oct 30 13:32:09.064597 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:32:09.064567    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-c78854-59f4c49558-vfdh9"]
Oct 30 13:32:09.065455 ip-10-0-142-84 kubenswrapper[4943]: W1030 13:32:09.065429    4943 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0b7cc484_1df0_4838_8bac_ad5229817bf8.slice/crio-f94010cf15341be4aaacb7b8a4d7e95671073dc5eb4dd6d72eecd918724d1078 WatchSource:0}: Error finding container f94010cf15341be4aaacb7b8a4d7e95671073dc5eb4dd6d72eecd918724d1078: Status 404 returned error can't find the container with id f94010cf15341be4aaacb7b8a4d7e95671073dc5eb4dd6d72eecd918724d1078
Oct 30 13:32:09.672014 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:32:09.671977    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-c78854-59f4c49558-vfdh9" event={"ID":"0b7cc484-1df0-4838-8bac-ad5229817bf8","Type":"ContainerStarted","Data":"c9a7e8dc8b44b18a4d0f6cef9f9c9b42aba8b685393185a66e071631a9d8fec4"}
Oct 30 13:32:09.672236 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:32:09.672074    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-c78854-59f4c49558-vfdh9" event={"ID":"0b7cc484-1df0-4838-8bac-ad5229817bf8","Type":"ContainerStarted","Data":"f94010cf15341be4aaacb7b8a4d7e95671073dc5eb4dd6d72eecd918724d1078"}
Oct 30 13:32:09.672236 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:32:09.672106    4943 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-c78854-59f4c49558-vfdh9"
Oct 30 13:32:09.720580 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:32:09.720520    4943 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-c78854-59f4c49558-vfdh9" podStartSLOduration=2.284434933 podStartE2EDuration="2.720506455s" totalImagesPullingTime="436.071522ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:32:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:32:09.717548804 +0000 UTC m=+2105.505552533" watchObservedRunningTime="2026-10-30 13:32:09.720506455 +0000 UTC m=+2105.508510184"
Oct 30 13:32:11.551935 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:32:11.551891    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-hdmrm"]
Oct 30 13:32:15.683317 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:32:15.683233    4943 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-c78854-59f4c49558-vfdh9"
Oct 30 13:32:35.786185 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:32:35.786146    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-hdmrm"]
Oct 30 13:32:35.956631 ip-10-0-142-84 kubenswrapper[4943]: E1030 13:32:35.956587    4943 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-knqz6" containerName="extractor"
Oct 30 13:32:51.217937 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:32:51.217901    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-hdmrm"]
Oct 30 13:33:05.955210 ip-10-0-142-84 kubenswrapper[4943]: E1030 13:33:05.955171    4943 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-knqz6" containerName="extractor"
Oct 30 13:33:12.527352 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:33:12.527315    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-hdmrm"]
Oct 30 13:33:35.955479 ip-10-0-142-84 kubenswrapper[4943]: E1030 13:33:35.955435    4943 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-knqz6" containerName="extractor"
Oct 30 13:33:36.993721 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:33:36.993679    4943 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-c78854-59f4c49558-vfdh9"]
Oct 30 13:33:36.994303 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:33:36.993956    4943 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-c78854-59f4c49558-vfdh9" podUID="0b7cc484-1df0-4838-8bac-ad5229817bf8" containerName="maas-api" containerID="cri-o://c9a7e8dc8b44b18a4d0f6cef9f9c9b42aba8b685393185a66e071631a9d8fec4" gracePeriod=30
Oct 30 13:33:37.174754 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:33:37.174714    4943 generic.go:356] "Generic (PLEG): container finished" podID="0b7cc484-1df0-4838-8bac-ad5229817bf8" containerID="c9a7e8dc8b44b18a4d0f6cef9f9c9b42aba8b685393185a66e071631a9d8fec4" exitCode=0
Oct 30 13:33:37.174930 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:33:37.174850    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-c78854-59f4c49558-vfdh9" event={"ID":"0b7cc484-1df0-4838-8bac-ad5229817bf8","Type":"ContainerDied","Data":"c9a7e8dc8b44b18a4d0f6cef9f9c9b42aba8b685393185a66e071631a9d8fec4"}
Oct 30 13:33:37.247642 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:33:37.247574    4943 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-c78854-59f4c49558-vfdh9"
Oct 30 13:33:37.396222 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:33:37.396191    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0b7cc484-1df0-4838-8bac-ad5229817bf8-kube-api-access-6p5gh\" (UniqueName: \"kubernetes.io/projected/0b7cc484-1df0-4838-8bac-ad5229817bf8-kube-api-access-6p5gh\") pod \"0b7cc484-1df0-4838-8bac-ad5229817bf8\" (UID: \"0b7cc484-1df0-4838-8bac-ad5229817bf8\") "
Oct 30 13:33:37.396222 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:33:37.396233    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/0b7cc484-1df0-4838-8bac-ad5229817bf8-metrics-certs\" (UniqueName: \"kubernetes.io/secret/0b7cc484-1df0-4838-8bac-ad5229817bf8-metrics-certs\") pod \"0b7cc484-1df0-4838-8bac-ad5229817bf8\" (UID: \"0b7cc484-1df0-4838-8bac-ad5229817bf8\") "
Oct 30 13:33:37.396478 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:33:37.396334    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0b7cc484-1df0-4838-8bac-ad5229817bf8-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0b7cc484-1df0-4838-8bac-ad5229817bf8-openshift-service-ca\") pod \"0b7cc484-1df0-4838-8bac-ad5229817bf8\" (UID: \"0b7cc484-1df0-4838-8bac-ad5229817bf8\") "
Oct 30 13:33:37.396478 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:33:37.396397    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/0b7cc484-1df0-4838-8bac-ad5229817bf8-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0b7cc484-1df0-4838-8bac-ad5229817bf8-maas-api-tls\") pod \"0b7cc484-1df0-4838-8bac-ad5229817bf8\" (UID: \"0b7cc484-1df0-4838-8bac-ad5229817bf8\") "
Oct 30 13:33:37.396799 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:33:37.396755    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0b7cc484-1df0-4838-8bac-ad5229817bf8-openshift-service-ca" pod "0b7cc484-1df0-4838-8bac-ad5229817bf8" (UID: "0b7cc484-1df0-4838-8bac-ad5229817bf8"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 13:33:37.398501 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:33:37.398462    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0b7cc484-1df0-4838-8bac-ad5229817bf8-metrics-certs" pod "0b7cc484-1df0-4838-8bac-ad5229817bf8" (UID: "0b7cc484-1df0-4838-8bac-ad5229817bf8"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 13:33:37.398861 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:33:37.398838    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0b7cc484-1df0-4838-8bac-ad5229817bf8-maas-api-tls" pod "0b7cc484-1df0-4838-8bac-ad5229817bf8" (UID: "0b7cc484-1df0-4838-8bac-ad5229817bf8"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 13:33:37.398930 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:33:37.398871    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0b7cc484-1df0-4838-8bac-ad5229817bf8-kube-api-access-6p5gh" pod "0b7cc484-1df0-4838-8bac-ad5229817bf8" (UID: "0b7cc484-1df0-4838-8bac-ad5229817bf8"). InnerVolumeSpecName "kube-api-access-6p5gh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:33:37.497485 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:33:37.497440    4943 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6p5gh\" (UniqueName: \"kubernetes.io/projected/0b7cc484-1df0-4838-8bac-ad5229817bf8-kube-api-access-6p5gh\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:33:37.497485 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:33:37.497475    4943 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0b7cc484-1df0-4838-8bac-ad5229817bf8-metrics-certs\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:33:37.497485 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:33:37.497489    4943 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0b7cc484-1df0-4838-8bac-ad5229817bf8-openshift-service-ca\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:33:37.497767 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:33:37.497502    4943 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0b7cc484-1df0-4838-8bac-ad5229817bf8-maas-api-tls\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:33:38.182214 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:33:38.182175    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-c78854-59f4c49558-vfdh9" event={"ID":"0b7cc484-1df0-4838-8bac-ad5229817bf8","Type":"ContainerDied","Data":"f94010cf15341be4aaacb7b8a4d7e95671073dc5eb4dd6d72eecd918724d1078"}
Oct 30 13:33:38.182214 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:33:38.182204    4943 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-c78854-59f4c49558-vfdh9"
Oct 30 13:33:38.182778 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:33:38.182237    4943 scope.go:122] "RemoveContainer" containerID="c9a7e8dc8b44b18a4d0f6cef9f9c9b42aba8b685393185a66e071631a9d8fec4"
Oct 30 13:33:38.257313 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:33:38.257278    4943 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-c78854-59f4c49558-vfdh9"]
Oct 30 13:33:38.271043 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:33:38.271003    4943 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-c78854-59f4c49558-vfdh9"]
Oct 30 13:33:38.353198 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:33:38.353167    4943 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0b7cc484-1df0-4838-8bac-ad5229817bf8" path="/var/lib/kubelet/pods/0b7cc484-1df0-4838-8bac-ad5229817bf8/volumes"
Oct 30 13:33:40.333429 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:33:40.333390    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-hdmrm"]
Oct 30 13:34:03.748559 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:34:03.748523    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-hdmrm"]
Oct 30 13:34:06.057283 ip-10-0-142-84 kubenswrapper[4943]: E1030 13:34:06.055546    4943 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-knqz6" containerName="extractor"
Oct 30 13:34:20.835849 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:34:20.835805    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-hdmrm"]
Oct 30 13:34:26.060995 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:34:26.060956    4943 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-87wwj"]
Oct 30 13:34:26.074587 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:34:26.074560    4943 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-87wwj"]
Oct 30 13:34:26.349493 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:34:26.349418    4943 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4207d02f-ec71-4f06-9d6b-75c7e43a45f8" path="/var/lib/kubelet/pods/4207d02f-ec71-4f06-9d6b-75c7e43a45f8/volumes"
Oct 30 13:34:35.955406 ip-10-0-142-84 kubenswrapper[4943]: E1030 13:34:35.955365    4943 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-knqz6" containerName="extractor"
Oct 30 13:34:37.223358 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:34:37.223321    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-hdmrm"]
Oct 30 13:34:49.199851 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:34:49.199810    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-hdmrm"]
Oct 30 13:34:56.333379 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:34:56.332539    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-hdmrm"]
Oct 30 13:35:04.685251 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:35:04.685212    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-hdmrm"]
Oct 30 13:35:06.087285 ip-10-0-142-84 kubenswrapper[4943]: E1030 13:35:06.087100    4943 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-knqz6" containerName="extractor"
Oct 30 13:35:18.187172 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:35:18.187089    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-hdmrm"]
Oct 30 13:35:35.108643 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:35:35.108605    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-hdmrm"]
Oct 30 13:35:35.956460 ip-10-0-142-84 kubenswrapper[4943]: E1030 13:35:35.956419    4943 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-knqz6" containerName="extractor"
Oct 30 13:35:36.208097 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:35:36.208021    4943 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-595cc648c-tt9fb"]
Oct 30 13:35:36.208607 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:35:36.208587    4943 cpu_manager.go:414] "containerMap: removing container" podUID="0b7cc484-1df0-4838-8bac-ad5229817bf8" containerName="maas-api"
Oct 30 13:35:36.208607 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:35:36.208608    4943 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0b7cc484-1df0-4838-8bac-ad5229817bf8" containerName="maas-api"
Oct 30 13:35:36.208736 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:35:36.208690    4943 memory_manager.go:365] "RemoveStaleState removing state" podUID="0b7cc484-1df0-4838-8bac-ad5229817bf8" containerName="maas-api"
Oct 30 13:35:36.211900 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:35:36.211881    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-595cc648c-tt9fb"
Oct 30 13:35:36.244312 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:35:36.244282    4943 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Oct 30 13:35:36.244312 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:35:36.244305    4943 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Oct 30 13:35:36.247132 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:35:36.247110    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-595cc648c-tt9fb"]
Oct 30 13:35:36.311447 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:35:36.311412    4943 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Oct 30 13:35:36.311617 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:35:36.311533    4943 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-5qrbm\""
Oct 30 13:35:36.311617 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:35:36.311556    4943 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Oct 30 13:35:36.391440 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:35:36.391408    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/431bcc86-68c2-498a-b889-6a78077b410c-metrics-certs\") pod \"maas-api-595cc648c-tt9fb\" (UID: \"431bcc86-68c2-498a-b889-6a78077b410c\") " pod="odh-ai-gateway-infra/maas-api-595cc648c-tt9fb"
Oct 30 13:35:36.391440 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:35:36.391440    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fn5rx\" (UniqueName: \"kubernetes.io/projected/431bcc86-68c2-498a-b889-6a78077b410c-kube-api-access-fn5rx\") pod \"maas-api-595cc648c-tt9fb\" (UID: \"431bcc86-68c2-498a-b889-6a78077b410c\") " pod="odh-ai-gateway-infra/maas-api-595cc648c-tt9fb"
Oct 30 13:35:36.391637 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:35:36.391617    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/431bcc86-68c2-498a-b889-6a78077b410c-openshift-service-ca\") pod \"maas-api-595cc648c-tt9fb\" (UID: \"431bcc86-68c2-498a-b889-6a78077b410c\") " pod="odh-ai-gateway-infra/maas-api-595cc648c-tt9fb"
Oct 30 13:35:36.391677 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:35:36.391647    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/431bcc86-68c2-498a-b889-6a78077b410c-maas-api-tls\") pod \"maas-api-595cc648c-tt9fb\" (UID: \"431bcc86-68c2-498a-b889-6a78077b410c\") " pod="odh-ai-gateway-infra/maas-api-595cc648c-tt9fb"
Oct 30 13:35:36.492910 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:35:36.492830    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/431bcc86-68c2-498a-b889-6a78077b410c-openshift-service-ca\") pod \"maas-api-595cc648c-tt9fb\" (UID: \"431bcc86-68c2-498a-b889-6a78077b410c\") " pod="odh-ai-gateway-infra/maas-api-595cc648c-tt9fb"
Oct 30 13:35:36.492910 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:35:36.492875    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/431bcc86-68c2-498a-b889-6a78077b410c-maas-api-tls\") pod \"maas-api-595cc648c-tt9fb\" (UID: \"431bcc86-68c2-498a-b889-6a78077b410c\") " pod="odh-ai-gateway-infra/maas-api-595cc648c-tt9fb"
Oct 30 13:35:36.493136 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:35:36.492930    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/431bcc86-68c2-498a-b889-6a78077b410c-metrics-certs\") pod \"maas-api-595cc648c-tt9fb\" (UID: \"431bcc86-68c2-498a-b889-6a78077b410c\") " pod="odh-ai-gateway-infra/maas-api-595cc648c-tt9fb"
Oct 30 13:35:36.493136 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:35:36.492949    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fn5rx\" (UniqueName: \"kubernetes.io/projected/431bcc86-68c2-498a-b889-6a78077b410c-kube-api-access-fn5rx\") pod \"maas-api-595cc648c-tt9fb\" (UID: \"431bcc86-68c2-498a-b889-6a78077b410c\") " pod="odh-ai-gateway-infra/maas-api-595cc648c-tt9fb"
Oct 30 13:35:36.493604 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:35:36.493577    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/431bcc86-68c2-498a-b889-6a78077b410c-openshift-service-ca\") pod \"maas-api-595cc648c-tt9fb\" (UID: \"431bcc86-68c2-498a-b889-6a78077b410c\") " pod="odh-ai-gateway-infra/maas-api-595cc648c-tt9fb"
Oct 30 13:35:36.495471 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:35:36.495452    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/431bcc86-68c2-498a-b889-6a78077b410c-metrics-certs\") pod \"maas-api-595cc648c-tt9fb\" (UID: \"431bcc86-68c2-498a-b889-6a78077b410c\") " pod="odh-ai-gateway-infra/maas-api-595cc648c-tt9fb"
Oct 30 13:35:36.495573 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:35:36.495501    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/431bcc86-68c2-498a-b889-6a78077b410c-maas-api-tls\") pod \"maas-api-595cc648c-tt9fb\" (UID: \"431bcc86-68c2-498a-b889-6a78077b410c\") " pod="odh-ai-gateway-infra/maas-api-595cc648c-tt9fb"
Oct 30 13:35:36.518447 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:35:36.518418    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fn5rx\" (UniqueName: \"kubernetes.io/projected/431bcc86-68c2-498a-b889-6a78077b410c-kube-api-access-fn5rx\") pod \"maas-api-595cc648c-tt9fb\" (UID: \"431bcc86-68c2-498a-b889-6a78077b410c\") " pod="odh-ai-gateway-infra/maas-api-595cc648c-tt9fb"
Oct 30 13:35:36.529091 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:35:36.529066    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-595cc648c-tt9fb"
Oct 30 13:35:36.834290 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:35:36.834249    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-595cc648c-tt9fb"]
Oct 30 13:35:36.835477 ip-10-0-142-84 kubenswrapper[4943]: W1030 13:35:36.835452    4943 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod431bcc86_68c2_498a_b889_6a78077b410c.slice/crio-8eb064336c8a816045c64aee734af5cab49cdd6c4ffda4ee7cbd7a429d2cce34 WatchSource:0}: Error finding container 8eb064336c8a816045c64aee734af5cab49cdd6c4ffda4ee7cbd7a429d2cce34: Status 404 returned error can't find the container with id 8eb064336c8a816045c64aee734af5cab49cdd6c4ffda4ee7cbd7a429d2cce34
Oct 30 13:35:36.836780 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:35:36.836762    4943 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 13:35:37.135940 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:35:37.135862    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-595cc648c-tt9fb" event={"ID":"431bcc86-68c2-498a-b889-6a78077b410c","Type":"ContainerStarted","Data":"8eb064336c8a816045c64aee734af5cab49cdd6c4ffda4ee7cbd7a429d2cce34"}
Oct 30 13:35:38.142756 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:35:38.142720    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-595cc648c-tt9fb" event={"ID":"431bcc86-68c2-498a-b889-6a78077b410c","Type":"ContainerStarted","Data":"526c530379f77088d62e021b3b85106979621676f42d60e7fe27366834d7ba60"}
Oct 30 13:35:38.143220 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:35:38.142802    4943 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-595cc648c-tt9fb"
Oct 30 13:35:38.193701 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:35:38.193649    4943 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-595cc648c-tt9fb" podStartSLOduration=1.803339284 podStartE2EDuration="2.193636437s" totalImagesPullingTime="390.297153ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:35:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:35:38.190523781 +0000 UTC m=+2313.978527511" watchObservedRunningTime="2026-10-30 13:35:38.193636437 +0000 UTC m=+2313.981640166"
Oct 30 13:35:40.015834 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:35:40.015800    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-hdmrm"]
Oct 30 13:35:44.156439 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:35:44.156406    4943 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-595cc648c-tt9fb"
Oct 30 13:35:58.183404 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:35:58.183368    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-hdmrm"]
Oct 30 13:36:00.968285 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:36:00.967406    4943 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-c9ffffc7f-q58qm container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 13:36:00.968285 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:36:00.967509    4943 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-q58qm" podUID="883f1e32-141f-45e4-9f0e-31f9231fab0d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:36:01.187306 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:36:01.186369    4943 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-c9ffffc7f-xs2vr container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.49:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 13:36:01.187306 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:36:01.186438    4943 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-xs2vr" podUID="e3af0d52-346e-48d7-ac7d-8f2aeee62448" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.49:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:36:05.954703 ip-10-0-142-84 kubenswrapper[4943]: E1030 13:36:05.954662    4943 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-knqz6" containerName="extractor"
Oct 30 13:36:08.211746 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:36:08.211714    4943 scope.go:122] "RemoveContainer" containerID="ba2ab005a3f9298354e558dc5f7d9ec9bbc55d96d3e9464a957c236c6aa77493"
Oct 30 13:36:16.285223 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:36:16.285186    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-hdmrm"]
Oct 30 13:36:35.954308 ip-10-0-142-84 kubenswrapper[4943]: E1030 13:36:35.954244    4943 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-knqz6" containerName="extractor"
Oct 30 13:36:44.396637 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:36:44.396600    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-hdmrm"]
Oct 30 13:36:44.799123 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:36:44.799088    4943 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-78df7c7756-pzxsg_bba72dc9-d1a2-4cf5-a205-4ee7250c7916/authorino/0.log"
Oct 30 13:36:45.984355 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:36:45.983379    4943 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-c9ffffc7f-q58qm container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 13:36:45.984355 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:36:45.983466    4943 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-q58qm" podUID="883f1e32-141f-45e4-9f0e-31f9231fab0d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:36:46.179317 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:36:46.173421    4943 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-c9ffffc7f-xs2vr container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.49:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 13:36:46.179317 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:36:46.173530    4943 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-xs2vr" podUID="e3af0d52-346e-48d7-ac7d-8f2aeee62448" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.49:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:37:04.660927 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:37:04.660899    4943 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ajkfrv_17f5a8d2-67e0-4489-8734-00f0d4483788/extract/0.log"
Oct 30 13:37:04.672025 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:37:04.671998    4943 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-samples-operator_cluster-samples-operator-6ddb6c8b94-7dtcd_23297e74-48f9-47b7-bb5a-80189bdbf2c4/cluster-samples-operator/0.log"
Oct 30 13:37:04.674067 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:37:04.674045    4943 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-8fqfr_d8930ee7-0ae0-48b5-8ce3-cb8820355cef/console-operator/2.log"
Oct 30 13:37:04.679139 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:37:04.679114    4943 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ajkfrv_17f5a8d2-67e0-4489-8734-00f0d4483788/extract/0.log"
Oct 30 13:37:04.689134 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:37:04.689108    4943 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-samples-operator_cluster-samples-operator-6ddb6c8b94-7dtcd_23297e74-48f9-47b7-bb5a-80189bdbf2c4/cluster-samples-operator/0.log"
Oct 30 13:37:04.691354 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:37:04.691332    4943 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-8fqfr_d8930ee7-0ae0-48b5-8ce3-cb8820355cef/console-operator/2.log"
Oct 30 13:37:05.956135 ip-10-0-142-84 kubenswrapper[4943]: E1030 13:37:05.956101    4943 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-knqz6" containerName="extractor"
Oct 30 13:37:35.933534 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:37:35.933500    4943 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_ai-gateway-controller-976c7b7d8-tlndh_5c697467-45ca-4c0e-88e4-32e23cb6c2dd/manager/0.log"
Oct 30 13:37:35.956633 ip-10-0-142-84 kubenswrapper[4943]: E1030 13:37:35.956600    4943 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-knqz6" containerName="extractor"
Oct 30 13:37:36.425577 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:37:36.425506    4943 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-module-controller-manager-5cfb77bf8c-8r9mm_6eb5db74-9830-41a6-8778-7e7e68213a8a/manager/0.log"
Oct 30 13:37:36.925324 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:37:36.925288    4943 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-5444d7fdb7-jhc74_0d971100-57cf-4513-9ca7-1f9536f3d842/manager/0.log"
Oct 30 13:37:37.573995 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:37:37.573967    4943 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-observability-584f99c788-mwqh2_310f7577-cbbe-4321-98eb-6d65bdc9cad2/manager/0.log"
Oct 30 13:37:38.080114 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:37:38.080087    4943 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-64d4d4766b-5rmd7_0d20cc9a-79da-4271-9b8e-8d667fd813ba/copy-manifests/0.log"
Oct 30 13:37:38.116712 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:37:38.116683    4943 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-64d4d4766b-5rmd7_0d20cc9a-79da-4271-9b8e-8d667fd813ba/manager/0.log"
Oct 30 13:37:49.625979 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:37:49.625937    4943 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-595cc648c-tt9fb_431bcc86-68c2-498a-b889-6a78077b410c/maas-api/0.log"
Oct 30 13:38:05.956067 ip-10-0-142-84 kubenswrapper[4943]: E1030 13:38:05.956017    4943 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-knqz6" containerName="extractor"
Oct 30 13:38:12.838566 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:38:12.838531    4943 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-78df7c7756-pzxsg_bba72dc9-d1a2-4cf5-a205-4ee7250c7916/authorino/0.log"
Oct 30 13:38:13.118194 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:38:13.118118    4943 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-68d6dd76dc-8fv2z_911d5399-ef35-4341-adb3-5d3de9c7b5fb/manager/0.log"
Oct 30 13:38:13.959829 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:38:13.959801    4943 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-7c9b87d574-hdmrm_306df0eb-65a9-4f3f-b55b-fb4389c208e8/limitador/0.log"
Oct 30 13:38:14.157393 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:38:14.157362    4943 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-856dd584c5-hlq89_a303f0b6-1d9d-46a1-9309-43e09bec7618/manager/0.log"
Oct 30 13:38:15.164632 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:38:15.164594    4943 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-7ddd9cfdc8-vvkvh_1d6ac649-474a-4e72-b202-259df21727a5/discovery/0.log"
Oct 30 13:38:15.364326 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:38:15.364292    4943 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-75fdd59bb7-cpmzt_e4cc43ab-deda-4426-83f5-236f95bfdfc4/kube-auth-proxy/0.log"
Oct 30 13:38:15.752996 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:38:15.752967    4943 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-c9ffffc7f-6h9t4_acf3c9ea-1f3e-40d5-88cd-2b76d8897517/istio-proxy/0.log"
Oct 30 13:38:26.346673 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:38:26.346625    4943 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-c9ffffc7f-pd5gs_28d7688e-c812-4fcc-b223-151368afe767/istio-proxy/0.log"
Oct 30 13:38:26.755955 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:38:26.755922    4943 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-c9ffffc7f-q58qm_883f1e32-141f-45e4-9f0e-31f9231fab0d/istio-proxy/0.log"
Oct 30 13:38:27.195608 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:38:27.195565    4943 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-c9ffffc7f-xs2vr_e3af0d52-346e-48d7-ac7d-8f2aeee62448/istio-proxy/0.log"
Oct 30 13:38:27.816554 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:38:27.816520    4943 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7b98bb678c-666x8_94a883ed-7c63-402d-bf2e-2e4ce2383175/payload-pre-processing/0.log"
Oct 30 13:38:28.014760 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:38:28.014736    4943 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5f78687d5b-r2pcb_7d0643d5-c375-42a2-ab82-f90aa05ec22d/payload-processing/0.log"
Oct 30 13:38:28.227694 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:38:28.227668    4943 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-649bdc978f-lkpqz_e1f629e6-6a96-444f-8329-e71bb98c9029/router/0.log"
Oct 30 13:38:29.870987 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:38:29.870954    4943 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-69b588897d-jt9hf_1d868c8e-735b-4008-a48f-d8311c5d31c5/main/0.log"
Oct 30 13:38:30.078542 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:38:30.078511    4943 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8kv25h_de3d7750-cd0c-4fbf-9ccf-b061d5d20e04/main/0.log"
Oct 30 13:38:35.954985 ip-10-0-142-84 kubenswrapper[4943]: E1030 13:38:35.954945    4943 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-knqz6" containerName="extractor"
Oct 30 13:38:41.431982 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:38:41.431948    4943 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-78df7c7756-pzxsg_bba72dc9-d1a2-4cf5-a205-4ee7250c7916/authorino/0.log"
Oct 30 13:39:00.982540 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:39:00.982512    4943 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_ai-gateway-controller-976c7b7d8-tlndh_5c697467-45ca-4c0e-88e4-32e23cb6c2dd/manager/0.log"
Oct 30 13:39:01.407746 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:39:01.407659    4943 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-module-controller-manager-5cfb77bf8c-8r9mm_6eb5db74-9830-41a6-8778-7e7e68213a8a/manager/0.log"
Oct 30 13:39:01.886085 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:39:01.886050    4943 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-5444d7fdb7-jhc74_0d971100-57cf-4513-9ca7-1f9536f3d842/manager/0.log"
Oct 30 13:39:02.560367 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:39:02.560331    4943 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-observability-584f99c788-mwqh2_310f7577-cbbe-4321-98eb-6d65bdc9cad2/manager/0.log"
Oct 30 13:39:03.043471 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:39:03.043435    4943 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-64d4d4766b-5rmd7_0d20cc9a-79da-4271-9b8e-8d667fd813ba/copy-manifests/0.log"
Oct 30 13:39:03.081006 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:39:03.080961    4943 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-64d4d4766b-5rmd7_0d20cc9a-79da-4271-9b8e-8d667fd813ba/manager/0.log"
Oct 30 13:39:04.334936 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:39:04.334906    4943 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-595cc648c-tt9fb_431bcc86-68c2-498a-b889-6a78077b410c/maas-api/0.log"
Oct 30 13:39:05.954213 ip-10-0-142-84 kubenswrapper[4943]: E1030 13:39:05.954179    4943 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-knqz6" containerName="extractor"
Oct 30 13:39:16.998056 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:39:16.998024    4943 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-78df7c7756-pzxsg_bba72dc9-d1a2-4cf5-a205-4ee7250c7916/authorino/0.log"
Oct 30 13:39:17.296335 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:39:17.296245    4943 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-68d6dd76dc-8fv2z_911d5399-ef35-4341-adb3-5d3de9c7b5fb/manager/0.log"
Oct 30 13:39:28.161407 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:39:28.161376    4943 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-7c9b87d574-hdmrm_306df0eb-65a9-4f3f-b55b-fb4389c208e8/limitador/0.log"
Oct 30 13:39:28.364951 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:39:28.364915    4943 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-856dd584c5-hlq89_a303f0b6-1d9d-46a1-9309-43e09bec7618/manager/0.log"
Oct 30 13:39:29.208667 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:39:29.208639    4943 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-7ddd9cfdc8-vvkvh_1d6ac649-474a-4e72-b202-259df21727a5/discovery/0.log"
Oct 30 13:39:29.404438 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:39:29.404406    4943 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-75fdd59bb7-cpmzt_e4cc43ab-deda-4426-83f5-236f95bfdfc4/kube-auth-proxy/0.log"
Oct 30 13:39:29.799975 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:39:29.799945    4943 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-c9ffffc7f-6h9t4_acf3c9ea-1f3e-40d5-88cd-2b76d8897517/istio-proxy/0.log"
Oct 30 13:39:30.407056 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:39:30.407023    4943 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-c9ffffc7f-pd5gs_28d7688e-c812-4fcc-b223-151368afe767/istio-proxy/0.log"
Oct 30 13:39:30.800971 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:39:30.800942    4943 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-c9ffffc7f-q58qm_883f1e32-141f-45e4-9f0e-31f9231fab0d/istio-proxy/0.log"
Oct 30 13:39:31.205599 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:39:31.205565    4943 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-c9ffffc7f-xs2vr_e3af0d52-346e-48d7-ac7d-8f2aeee62448/istio-proxy/0.log"
Oct 30 13:39:31.800486 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:39:31.800451    4943 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7b98bb678c-666x8_94a883ed-7c63-402d-bf2e-2e4ce2383175/payload-pre-processing/0.log"
Oct 30 13:39:31.996618 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:39:31.996585    4943 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5f78687d5b-r2pcb_7d0643d5-c375-42a2-ab82-f90aa05ec22d/payload-processing/0.log"
Oct 30 13:39:32.191084 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:39:32.191055    4943 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-649bdc978f-lkpqz_e1f629e6-6a96-444f-8329-e71bb98c9029/router/0.log"
Oct 30 13:39:33.336486 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:39:33.336454    4943 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-69b588897d-jt9hf_1d868c8e-735b-4008-a48f-d8311c5d31c5/main/0.log"
Oct 30 13:39:33.549320 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:39:33.549287    4943 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8kv25h_de3d7750-cd0c-4fbf-9ccf-b061d5d20e04/main/0.log"
Oct 30 13:39:35.955206 ip-10-0-142-84 kubenswrapper[4943]: E1030 13:39:35.955170    4943 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-knqz6" containerName="extractor"
Oct 30 13:40:05.953806 ip-10-0-142-84 kubenswrapper[4943]: E1030 13:40:05.953716    4943 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-knqz6" containerName="extractor"
Oct 30 13:40:35.954630 ip-10-0-142-84 kubenswrapper[4943]: E1030 13:40:35.954594    4943 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-knqz6" containerName="extractor"
Oct 30 13:40:53.507657 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:40:53.507624    4943 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-5444d7fdb7-jhc74_0d971100-57cf-4513-9ca7-1f9536f3d842/manager/0.log"
Oct 30 13:40:53.850810 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:40:53.850731    4943 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-5444d7fdb7-jhc74_0d971100-57cf-4513-9ca7-1f9536f3d842/manager/0.log"
Oct 30 13:40:54.603907 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:40:54.603876    4943 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5f78687d5b-r2pcb_7d0643d5-c375-42a2-ab82-f90aa05ec22d/payload-processing/0.log"
Oct 30 13:40:54.796858 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:40:54.796822    4943 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5f78687d5b-r2pcb_7d0643d5-c375-42a2-ab82-f90aa05ec22d/payload-processing/0.log"
Oct 30 13:41:05.954674 ip-10-0-142-84 kubenswrapper[4943]: E1030 13:41:05.954635    4943 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-knqz6" containerName="extractor"
Oct 30 13:41:35.955077 ip-10-0-142-84 kubenswrapper[4943]: E1030 13:41:35.954995    4943 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-knqz6" containerName="extractor"
Oct 30 13:42:02.947241 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:42:02.947200    4943 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-5444d7fdb7-jhc74_0d971100-57cf-4513-9ca7-1f9536f3d842/manager/0.log"
Oct 30 13:42:03.261048 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:42:03.260967    4943 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-5444d7fdb7-jhc74_0d971100-57cf-4513-9ca7-1f9536f3d842/manager/0.log"
Oct 30 13:42:04.015918 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:42:04.015885    4943 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5f78687d5b-r2pcb_7d0643d5-c375-42a2-ab82-f90aa05ec22d/payload-processing/0.log"
Oct 30 13:42:04.209182 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:42:04.209146    4943 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5f78687d5b-r2pcb_7d0643d5-c375-42a2-ab82-f90aa05ec22d/payload-processing/0.log"
Oct 30 13:42:04.700494 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:42:04.700466    4943 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ajkfrv_17f5a8d2-67e0-4489-8734-00f0d4483788/extract/0.log"
Oct 30 13:42:04.710928 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:42:04.710904    4943 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-samples-operator_cluster-samples-operator-6ddb6c8b94-7dtcd_23297e74-48f9-47b7-bb5a-80189bdbf2c4/cluster-samples-operator/0.log"
Oct 30 13:42:04.713094 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:42:04.713067    4943 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-8fqfr_d8930ee7-0ae0-48b5-8ce3-cb8820355cef/console-operator/2.log"
Oct 30 13:42:04.719717 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:42:04.719688    4943 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ajkfrv_17f5a8d2-67e0-4489-8734-00f0d4483788/extract/0.log"
Oct 30 13:42:04.729444 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:42:04.729426    4943 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-samples-operator_cluster-samples-operator-6ddb6c8b94-7dtcd_23297e74-48f9-47b7-bb5a-80189bdbf2c4/cluster-samples-operator/0.log"
Oct 30 13:42:04.731497 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:42:04.731481    4943 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-8fqfr_d8930ee7-0ae0-48b5-8ce3-cb8820355cef/console-operator/2.log"
Oct 30 13:42:05.954939 ip-10-0-142-84 kubenswrapper[4943]: E1030 13:42:05.954901    4943 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-knqz6" containerName="extractor"
Oct 30 13:42:09.362089 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:42:09.362055    4943 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-rpdcd_f660a256-a12d-4a4a-823d-4b0ff5dcb3da/global-pull-secret-syncer/0.log"
Oct 30 13:42:09.446169 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:42:09.446138    4943 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-q58qm"]
Oct 30 13:42:09.446387 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:42:09.446368    4943 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-q58qm" podUID="883f1e32-141f-45e4-9f0e-31f9231fab0d" containerName="istio-proxy" containerID="cri-o://6c1e5ecd734fa6354f717ef54dd3d773e0452a91a0e6fba515a37fea89df216a" gracePeriod=30
Oct 30 13:42:09.460301 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:42:09.460254    4943 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-6h9t4"]
Oct 30 13:42:09.460533 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:42:09.460488    4943 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-6h9t4" podUID="acf3c9ea-1f3e-40d5-88cd-2b76d8897517" containerName="istio-proxy" containerID="cri-o://5b917183faf75df89f208e6d23a90615443ffea090d4ed8d6b13b68ec8ef840d" gracePeriod=30
Oct 30 13:42:09.462797 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:42:09.462763    4943 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-xs2vr"]
Oct 30 13:42:09.463066 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:42:09.463029    4943 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-xs2vr" podUID="e3af0d52-346e-48d7-ac7d-8f2aeee62448" containerName="istio-proxy" containerID="cri-o://0033fcc20732bb67a1bf7c2e2cf5e50511434b88f23ad93d8bff99cde7b389c6" gracePeriod=30
Oct 30 13:42:09.476841 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:42:09.476787    4943 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-pd5gs"]
Oct 30 13:42:09.477076 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:42:09.477049    4943 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-pd5gs" podUID="28d7688e-c812-4fcc-b223-151368afe767" containerName="istio-proxy" containerID="cri-o://c2ea5fd9167d5f62257af2c8b088a96935f2de64527cc69155eed73c02633fc6" gracePeriod=30
Oct 30 13:42:09.546588 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:42:09.546550    4943 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kas-connection-checker-685895c7c9-4zl6m_a4d58318-42ba-4a9e-9c74-43fe90baa4a7/connection-checker/0.log"
Oct 30 13:42:09.665871 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:42:09.665792    4943 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kas-connection-checker-685895c7c9-h7brb_301eebcb-3d48-4eef-a0de-a1173e23ad65/connection-checker/0.log"
Oct 30 13:42:09.788059 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:42:09.788028    4943 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kas-connection-checker-685895c7c9-k4v7j_0905782c-26ad-4dc8-93cf-d6f211ea7af5/connection-checker/0.log"
Oct 30 13:42:10.244488 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:42:10.244460    4943 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-zth9k_51d01732-87a4-42b0-a6d5-a3fac689d7f6/konnectivity-agent/0.log"
Oct 30 13:42:10.712252 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:42:10.712223    4943 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-142-84.ec2.internal_0c9cf3ec0e3ab50517e6f80b07fb5fb2/haproxy/0.log"
Oct 30 13:42:14.750408 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:42:14.750378    4943 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-q58qm"
Oct 30 13:42:14.779851 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:42:14.779816    4943 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-xs2vr"
Oct 30 13:42:14.810378 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:42:14.810343    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/883f1e32-141f-45e4-9f0e-31f9231fab0d-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/883f1e32-141f-45e4-9f0e-31f9231fab0d-istio-data\") pod \"883f1e32-141f-45e4-9f0e-31f9231fab0d\" (UID: \"883f1e32-141f-45e4-9f0e-31f9231fab0d\") "
Oct 30 13:42:14.810584 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:42:14.810425    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/883f1e32-141f-45e4-9f0e-31f9231fab0d-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/883f1e32-141f-45e4-9f0e-31f9231fab0d-workload-socket\") pod \"883f1e32-141f-45e4-9f0e-31f9231fab0d\" (UID: \"883f1e32-141f-45e4-9f0e-31f9231fab0d\") "
Oct 30 13:42:14.810584 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:42:14.810456    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/883f1e32-141f-45e4-9f0e-31f9231fab0d-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/883f1e32-141f-45e4-9f0e-31f9231fab0d-credential-socket\") pod \"883f1e32-141f-45e4-9f0e-31f9231fab0d\" (UID: \"883f1e32-141f-45e4-9f0e-31f9231fab0d\") "
Oct 30 13:42:14.810584 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:42:14.810482    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/883f1e32-141f-45e4-9f0e-31f9231fab0d-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/883f1e32-141f-45e4-9f0e-31f9231fab0d-workload-certs\") pod \"883f1e32-141f-45e4-9f0e-31f9231fab0d\" (UID: \"883f1e32-141f-45e4-9f0e-31f9231fab0d\") "
Oct 30 13:42:14.810584 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:42:14.810542    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/883f1e32-141f-45e4-9f0e-31f9231fab0d-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/883f1e32-141f-45e4-9f0e-31f9231fab0d-istio-envoy\") pod \"883f1e32-141f-45e4-9f0e-31f9231fab0d\" (UID: \"883f1e32-141f-45e4-9f0e-31f9231fab0d\") "
Oct 30 13:42:14.810584 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:42:14.810578    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/883f1e32-141f-45e4-9f0e-31f9231fab0d-istio-token\" (UniqueName: \"kubernetes.io/projected/883f1e32-141f-45e4-9f0e-31f9231fab0d-istio-token\") pod \"883f1e32-141f-45e4-9f0e-31f9231fab0d\" (UID: \"883f1e32-141f-45e4-9f0e-31f9231fab0d\") "
Oct 30 13:42:14.810873 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:42:14.810616    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/883f1e32-141f-45e4-9f0e-31f9231fab0d-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/883f1e32-141f-45e4-9f0e-31f9231fab0d-istio-podinfo\") pod \"883f1e32-141f-45e4-9f0e-31f9231fab0d\" (UID: \"883f1e32-141f-45e4-9f0e-31f9231fab0d\") "
Oct 30 13:42:14.810873 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:42:14.810645    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/883f1e32-141f-45e4-9f0e-31f9231fab0d-istio-data" pod "883f1e32-141f-45e4-9f0e-31f9231fab0d" (UID: "883f1e32-141f-45e4-9f0e-31f9231fab0d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:42:14.810873 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:42:14.810666    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/883f1e32-141f-45e4-9f0e-31f9231fab0d-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/883f1e32-141f-45e4-9f0e-31f9231fab0d-istiod-ca-cert\") pod \"883f1e32-141f-45e4-9f0e-31f9231fab0d\" (UID: \"883f1e32-141f-45e4-9f0e-31f9231fab0d\") "
Oct 30 13:42:14.810873 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:42:14.810731    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/883f1e32-141f-45e4-9f0e-31f9231fab0d-workload-socket" pod "883f1e32-141f-45e4-9f0e-31f9231fab0d" (UID: "883f1e32-141f-45e4-9f0e-31f9231fab0d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:42:14.810873 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:42:14.810760    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/883f1e32-141f-45e4-9f0e-31f9231fab0d-kube-api-access-gx7cn\" (UniqueName: \"kubernetes.io/projected/883f1e32-141f-45e4-9f0e-31f9231fab0d-kube-api-access-gx7cn\") pod \"883f1e32-141f-45e4-9f0e-31f9231fab0d\" (UID: \"883f1e32-141f-45e4-9f0e-31f9231fab0d\") "
Oct 30 13:42:14.810873 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:42:14.810843    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/883f1e32-141f-45e4-9f0e-31f9231fab0d-credential-socket" pod "883f1e32-141f-45e4-9f0e-31f9231fab0d" (UID: "883f1e32-141f-45e4-9f0e-31f9231fab0d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:42:14.811190 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:42:14.810922    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/883f1e32-141f-45e4-9f0e-31f9231fab0d-workload-certs" pod "883f1e32-141f-45e4-9f0e-31f9231fab0d" (UID: "883f1e32-141f-45e4-9f0e-31f9231fab0d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:42:14.811190 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:42:14.811132    4943 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/883f1e32-141f-45e4-9f0e-31f9231fab0d-workload-socket\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:42:14.811190 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:42:14.811155    4943 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/883f1e32-141f-45e4-9f0e-31f9231fab0d-credential-socket\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:42:14.811190 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:42:14.811171    4943 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/883f1e32-141f-45e4-9f0e-31f9231fab0d-workload-certs\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:42:14.811190 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:42:14.811186    4943 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/883f1e32-141f-45e4-9f0e-31f9231fab0d-istio-data\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:42:14.811498 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:42:14.811280    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/883f1e32-141f-45e4-9f0e-31f9231fab0d-istiod-ca-cert" pod "883f1e32-141f-45e4-9f0e-31f9231fab0d" (UID: "883f1e32-141f-45e4-9f0e-31f9231fab0d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 13:42:14.814083 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:42:14.814041    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/883f1e32-141f-45e4-9f0e-31f9231fab0d-istio-envoy" pod "883f1e32-141f-45e4-9f0e-31f9231fab0d" (UID: "883f1e32-141f-45e4-9f0e-31f9231fab0d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:42:14.815371 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:42:14.815346    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/883f1e32-141f-45e4-9f0e-31f9231fab0d-istio-podinfo" pod "883f1e32-141f-45e4-9f0e-31f9231fab0d" (UID: "883f1e32-141f-45e4-9f0e-31f9231fab0d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 13:42:14.815807 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:42:14.815713    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/883f1e32-141f-45e4-9f0e-31f9231fab0d-istio-token" pod "883f1e32-141f-45e4-9f0e-31f9231fab0d" (UID: "883f1e32-141f-45e4-9f0e-31f9231fab0d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:42:14.817920 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:42:14.817879    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/883f1e32-141f-45e4-9f0e-31f9231fab0d-kube-api-access-gx7cn" pod "883f1e32-141f-45e4-9f0e-31f9231fab0d" (UID: "883f1e32-141f-45e4-9f0e-31f9231fab0d"). InnerVolumeSpecName "kube-api-access-gx7cn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:42:14.830638 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:42:14.830605    4943 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-6h9t4"
Oct 30 13:42:14.839711 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:42:14.839686    4943 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-pd5gs"
Oct 30 13:42:14.912050 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:42:14.911998    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e3af0d52-346e-48d7-ac7d-8f2aeee62448-kube-api-access-55tg9\" (UniqueName: \"kubernetes.io/projected/e3af0d52-346e-48d7-ac7d-8f2aeee62448-kube-api-access-55tg9\") pod \"e3af0d52-346e-48d7-ac7d-8f2aeee62448\" (UID: \"e3af0d52-346e-48d7-ac7d-8f2aeee62448\") "
Oct 30 13:42:14.912050 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:42:14.912051    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/28d7688e-c812-4fcc-b223-151368afe767-kube-api-access-bqk9t\" (UniqueName: \"kubernetes.io/projected/28d7688e-c812-4fcc-b223-151368afe767-kube-api-access-bqk9t\") pod \"28d7688e-c812-4fcc-b223-151368afe767\" (UID: \"28d7688e-c812-4fcc-b223-151368afe767\") "
Oct 30 13:42:14.912425 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:42:14.912092    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/28d7688e-c812-4fcc-b223-151368afe767-istio-token\" (UniqueName: \"kubernetes.io/projected/28d7688e-c812-4fcc-b223-151368afe767-istio-token\") pod \"28d7688e-c812-4fcc-b223-151368afe767\" (UID: \"28d7688e-c812-4fcc-b223-151368afe767\") "
Oct 30 13:42:14.912425 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:42:14.912123    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/acf3c9ea-1f3e-40d5-88cd-2b76d8897517-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/acf3c9ea-1f3e-40d5-88cd-2b76d8897517-istio-data\") pod \"acf3c9ea-1f3e-40d5-88cd-2b76d8897517\" (UID: \"acf3c9ea-1f3e-40d5-88cd-2b76d8897517\") "
Oct 30 13:42:14.912425 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:42:14.912145    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e3af0d52-346e-48d7-ac7d-8f2aeee62448-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e3af0d52-346e-48d7-ac7d-8f2aeee62448-workload-certs\") pod \"e3af0d52-346e-48d7-ac7d-8f2aeee62448\" (UID: \"e3af0d52-346e-48d7-ac7d-8f2aeee62448\") "
Oct 30 13:42:14.912425 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:42:14.912162    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/acf3c9ea-1f3e-40d5-88cd-2b76d8897517-kube-api-access-rhwcd\" (UniqueName: \"kubernetes.io/projected/acf3c9ea-1f3e-40d5-88cd-2b76d8897517-kube-api-access-rhwcd\") pod \"acf3c9ea-1f3e-40d5-88cd-2b76d8897517\" (UID: \"acf3c9ea-1f3e-40d5-88cd-2b76d8897517\") "
Oct 30 13:42:14.912425 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:42:14.912191    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/28d7688e-c812-4fcc-b223-151368afe767-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/28d7688e-c812-4fcc-b223-151368afe767-istio-data\") pod \"28d7688e-c812-4fcc-b223-151368afe767\" (UID: \"28d7688e-c812-4fcc-b223-151368afe767\") "
Oct 30 13:42:14.912425 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:42:14.912220    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/e3af0d52-346e-48d7-ac7d-8f2aeee62448-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e3af0d52-346e-48d7-ac7d-8f2aeee62448-istio-podinfo\") pod \"e3af0d52-346e-48d7-ac7d-8f2aeee62448\" (UID: \"e3af0d52-346e-48d7-ac7d-8f2aeee62448\") "
Oct 30 13:42:14.912425 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:42:14.912242    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/acf3c9ea-1f3e-40d5-88cd-2b76d8897517-istio-token\" (UniqueName: \"kubernetes.io/projected/acf3c9ea-1f3e-40d5-88cd-2b76d8897517-istio-token\") pod \"acf3c9ea-1f3e-40d5-88cd-2b76d8897517\" (UID: \"acf3c9ea-1f3e-40d5-88cd-2b76d8897517\") "
Oct 30 13:42:14.912425 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:42:14.912297    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/acf3c9ea-1f3e-40d5-88cd-2b76d8897517-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/acf3c9ea-1f3e-40d5-88cd-2b76d8897517-workload-certs\") pod \"acf3c9ea-1f3e-40d5-88cd-2b76d8897517\" (UID: \"acf3c9ea-1f3e-40d5-88cd-2b76d8897517\") "
Oct 30 13:42:14.912425 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:42:14.912344    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/acf3c9ea-1f3e-40d5-88cd-2b76d8897517-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/acf3c9ea-1f3e-40d5-88cd-2b76d8897517-istio-podinfo\") pod \"acf3c9ea-1f3e-40d5-88cd-2b76d8897517\" (UID: \"acf3c9ea-1f3e-40d5-88cd-2b76d8897517\") "
Oct 30 13:42:14.912425 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:42:14.912384    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e3af0d52-346e-48d7-ac7d-8f2aeee62448-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e3af0d52-346e-48d7-ac7d-8f2aeee62448-istio-envoy\") pod \"e3af0d52-346e-48d7-ac7d-8f2aeee62448\" (UID: \"e3af0d52-346e-48d7-ac7d-8f2aeee62448\") "
Oct 30 13:42:14.912425 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:42:14.912407    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/acf3c9ea-1f3e-40d5-88cd-2b76d8897517-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/acf3c9ea-1f3e-40d5-88cd-2b76d8897517-istiod-ca-cert\") pod \"acf3c9ea-1f3e-40d5-88cd-2b76d8897517\" (UID: \"acf3c9ea-1f3e-40d5-88cd-2b76d8897517\") "
Oct 30 13:42:14.914510 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:42:14.912436    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/28d7688e-c812-4fcc-b223-151368afe767-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/28d7688e-c812-4fcc-b223-151368afe767-istio-podinfo\") pod \"28d7688e-c812-4fcc-b223-151368afe767\" (UID: \"28d7688e-c812-4fcc-b223-151368afe767\") "
Oct 30 13:42:14.914510 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:42:14.912494    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/acf3c9ea-1f3e-40d5-88cd-2b76d8897517-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/acf3c9ea-1f3e-40d5-88cd-2b76d8897517-istio-envoy\") pod \"acf3c9ea-1f3e-40d5-88cd-2b76d8897517\" (UID: \"acf3c9ea-1f3e-40d5-88cd-2b76d8897517\") "
Oct 30 13:42:14.914510 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:42:14.912576    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/28d7688e-c812-4fcc-b223-151368afe767-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/28d7688e-c812-4fcc-b223-151368afe767-istiod-ca-cert\") pod \"28d7688e-c812-4fcc-b223-151368afe767\" (UID: \"28d7688e-c812-4fcc-b223-151368afe767\") "
Oct 30 13:42:14.914510 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:42:14.912603    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e3af0d52-346e-48d7-ac7d-8f2aeee62448-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e3af0d52-346e-48d7-ac7d-8f2aeee62448-istio-data\") pod \"e3af0d52-346e-48d7-ac7d-8f2aeee62448\" (UID: \"e3af0d52-346e-48d7-ac7d-8f2aeee62448\") "
Oct 30 13:42:14.914510 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:42:14.912639    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/acf3c9ea-1f3e-40d5-88cd-2b76d8897517-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/acf3c9ea-1f3e-40d5-88cd-2b76d8897517-workload-socket\") pod \"acf3c9ea-1f3e-40d5-88cd-2b76d8897517\" (UID: \"acf3c9ea-1f3e-40d5-88cd-2b76d8897517\") "
Oct 30 13:42:14.914510 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:42:14.912680    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/28d7688e-c812-4fcc-b223-151368afe767-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/28d7688e-c812-4fcc-b223-151368afe767-workload-socket\") pod \"28d7688e-c812-4fcc-b223-151368afe767\" (UID: \"28d7688e-c812-4fcc-b223-151368afe767\") "
Oct 30 13:42:14.914510 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:42:14.912716    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/28d7688e-c812-4fcc-b223-151368afe767-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/28d7688e-c812-4fcc-b223-151368afe767-workload-certs\") pod \"28d7688e-c812-4fcc-b223-151368afe767\" (UID: \"28d7688e-c812-4fcc-b223-151368afe767\") "
Oct 30 13:42:14.914510 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:42:14.912748    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e3af0d52-346e-48d7-ac7d-8f2aeee62448-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e3af0d52-346e-48d7-ac7d-8f2aeee62448-istiod-ca-cert\") pod \"e3af0d52-346e-48d7-ac7d-8f2aeee62448\" (UID: \"e3af0d52-346e-48d7-ac7d-8f2aeee62448\") "
Oct 30 13:42:14.914510 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:42:14.912773    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e3af0d52-346e-48d7-ac7d-8f2aeee62448-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e3af0d52-346e-48d7-ac7d-8f2aeee62448-credential-socket\") pod \"e3af0d52-346e-48d7-ac7d-8f2aeee62448\" (UID: \"e3af0d52-346e-48d7-ac7d-8f2aeee62448\") "
Oct 30 13:42:14.914510 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:42:14.912797    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e3af0d52-346e-48d7-ac7d-8f2aeee62448-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e3af0d52-346e-48d7-ac7d-8f2aeee62448-workload-socket\") pod \"e3af0d52-346e-48d7-ac7d-8f2aeee62448\" (UID: \"e3af0d52-346e-48d7-ac7d-8f2aeee62448\") "
Oct 30 13:42:14.914510 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:42:14.912841    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e3af0d52-346e-48d7-ac7d-8f2aeee62448-istio-token\" (UniqueName: \"kubernetes.io/projected/e3af0d52-346e-48d7-ac7d-8f2aeee62448-istio-token\") pod \"e3af0d52-346e-48d7-ac7d-8f2aeee62448\" (UID: \"e3af0d52-346e-48d7-ac7d-8f2aeee62448\") "
Oct 30 13:42:14.914510 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:42:14.912871    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/acf3c9ea-1f3e-40d5-88cd-2b76d8897517-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/acf3c9ea-1f3e-40d5-88cd-2b76d8897517-credential-socket\") pod \"acf3c9ea-1f3e-40d5-88cd-2b76d8897517\" (UID: \"acf3c9ea-1f3e-40d5-88cd-2b76d8897517\") "
Oct 30 13:42:14.914510 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:42:14.912910    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/28d7688e-c812-4fcc-b223-151368afe767-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/28d7688e-c812-4fcc-b223-151368afe767-istio-envoy\") pod \"28d7688e-c812-4fcc-b223-151368afe767\" (UID: \"28d7688e-c812-4fcc-b223-151368afe767\") "
Oct 30 13:42:14.914510 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:42:14.912962    4943 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/28d7688e-c812-4fcc-b223-151368afe767-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/28d7688e-c812-4fcc-b223-151368afe767-credential-socket\") pod \"28d7688e-c812-4fcc-b223-151368afe767\" (UID: \"28d7688e-c812-4fcc-b223-151368afe767\") "
Oct 30 13:42:14.914510 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:42:14.913463    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e3af0d52-346e-48d7-ac7d-8f2aeee62448-istio-data" pod "e3af0d52-346e-48d7-ac7d-8f2aeee62448" (UID: "e3af0d52-346e-48d7-ac7d-8f2aeee62448"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:42:14.914510 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:42:14.913494    4943 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/883f1e32-141f-45e4-9f0e-31f9231fab0d-istiod-ca-cert\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:42:14.914510 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:42:14.913521    4943 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gx7cn\" (UniqueName: \"kubernetes.io/projected/883f1e32-141f-45e4-9f0e-31f9231fab0d-kube-api-access-gx7cn\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:42:14.914510 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:42:14.913539    4943 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/883f1e32-141f-45e4-9f0e-31f9231fab0d-istio-envoy\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:42:14.914510 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:42:14.913553    4943 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/883f1e32-141f-45e4-9f0e-31f9231fab0d-istio-token\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:42:14.914510 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:42:14.913566    4943 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/883f1e32-141f-45e4-9f0e-31f9231fab0d-istio-podinfo\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:42:14.914510 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:42:14.913776    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/28d7688e-c812-4fcc-b223-151368afe767-credential-socket" pod "28d7688e-c812-4fcc-b223-151368afe767" (UID: "28d7688e-c812-4fcc-b223-151368afe767"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:42:14.914510 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:42:14.914071    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/acf3c9ea-1f3e-40d5-88cd-2b76d8897517-credential-socket" pod "acf3c9ea-1f3e-40d5-88cd-2b76d8897517" (UID: "acf3c9ea-1f3e-40d5-88cd-2b76d8897517"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:42:14.914510 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:42:14.914204    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/acf3c9ea-1f3e-40d5-88cd-2b76d8897517-istio-data" pod "acf3c9ea-1f3e-40d5-88cd-2b76d8897517" (UID: "acf3c9ea-1f3e-40d5-88cd-2b76d8897517"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:42:14.914510 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:42:14.914364    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e3af0d52-346e-48d7-ac7d-8f2aeee62448-credential-socket" pod "e3af0d52-346e-48d7-ac7d-8f2aeee62448" (UID: "e3af0d52-346e-48d7-ac7d-8f2aeee62448"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:42:14.914510 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:42:14.914516    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/28d7688e-c812-4fcc-b223-151368afe767-workload-socket" pod "28d7688e-c812-4fcc-b223-151368afe767" (UID: "28d7688e-c812-4fcc-b223-151368afe767"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:42:14.915868 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:42:14.914643    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/28d7688e-c812-4fcc-b223-151368afe767-workload-certs" pod "28d7688e-c812-4fcc-b223-151368afe767" (UID: "28d7688e-c812-4fcc-b223-151368afe767"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:42:14.915868 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:42:14.914773    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/acf3c9ea-1f3e-40d5-88cd-2b76d8897517-workload-socket" pod "acf3c9ea-1f3e-40d5-88cd-2b76d8897517" (UID: "acf3c9ea-1f3e-40d5-88cd-2b76d8897517"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:42:14.915868 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:42:14.915104    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/acf3c9ea-1f3e-40d5-88cd-2b76d8897517-istiod-ca-cert" pod "acf3c9ea-1f3e-40d5-88cd-2b76d8897517" (UID: "acf3c9ea-1f3e-40d5-88cd-2b76d8897517"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 13:42:14.915868 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:42:14.915178    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e3af0d52-346e-48d7-ac7d-8f2aeee62448-istiod-ca-cert" pod "e3af0d52-346e-48d7-ac7d-8f2aeee62448" (UID: "e3af0d52-346e-48d7-ac7d-8f2aeee62448"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 13:42:14.915868 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:42:14.915333    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/acf3c9ea-1f3e-40d5-88cd-2b76d8897517-workload-certs" pod "acf3c9ea-1f3e-40d5-88cd-2b76d8897517" (UID: "acf3c9ea-1f3e-40d5-88cd-2b76d8897517"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:42:14.915868 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:42:14.915581    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e3af0d52-346e-48d7-ac7d-8f2aeee62448-workload-socket" pod "e3af0d52-346e-48d7-ac7d-8f2aeee62448" (UID: "e3af0d52-346e-48d7-ac7d-8f2aeee62448"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:42:14.916197 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:42:14.915966    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e3af0d52-346e-48d7-ac7d-8f2aeee62448-workload-certs" pod "e3af0d52-346e-48d7-ac7d-8f2aeee62448" (UID: "e3af0d52-346e-48d7-ac7d-8f2aeee62448"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:42:14.916840 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:42:14.916592    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e3af0d52-346e-48d7-ac7d-8f2aeee62448-kube-api-access-55tg9" pod "e3af0d52-346e-48d7-ac7d-8f2aeee62448" (UID: "e3af0d52-346e-48d7-ac7d-8f2aeee62448"). InnerVolumeSpecName "kube-api-access-55tg9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:42:14.916840 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:42:14.916709    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/acf3c9ea-1f3e-40d5-88cd-2b76d8897517-istio-token" pod "acf3c9ea-1f3e-40d5-88cd-2b76d8897517" (UID: "acf3c9ea-1f3e-40d5-88cd-2b76d8897517"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:42:14.916840 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:42:14.916787    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e3af0d52-346e-48d7-ac7d-8f2aeee62448-istio-podinfo" pod "e3af0d52-346e-48d7-ac7d-8f2aeee62448" (UID: "e3af0d52-346e-48d7-ac7d-8f2aeee62448"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 13:42:14.917121 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:42:14.916992    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/28d7688e-c812-4fcc-b223-151368afe767-istio-data" pod "28d7688e-c812-4fcc-b223-151368afe767" (UID: "28d7688e-c812-4fcc-b223-151368afe767"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:42:14.917389 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:42:14.917353    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/28d7688e-c812-4fcc-b223-151368afe767-istiod-ca-cert" pod "28d7688e-c812-4fcc-b223-151368afe767" (UID: "28d7688e-c812-4fcc-b223-151368afe767"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 13:42:14.919641 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:42:14.919614    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/28d7688e-c812-4fcc-b223-151368afe767-istio-envoy" pod "28d7688e-c812-4fcc-b223-151368afe767" (UID: "28d7688e-c812-4fcc-b223-151368afe767"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:42:14.919889 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:42:14.919854    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/acf3c9ea-1f3e-40d5-88cd-2b76d8897517-istio-podinfo" pod "acf3c9ea-1f3e-40d5-88cd-2b76d8897517" (UID: "acf3c9ea-1f3e-40d5-88cd-2b76d8897517"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 13:42:14.920058 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:42:14.920030    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/acf3c9ea-1f3e-40d5-88cd-2b76d8897517-istio-envoy" pod "acf3c9ea-1f3e-40d5-88cd-2b76d8897517" (UID: "acf3c9ea-1f3e-40d5-88cd-2b76d8897517"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:42:14.920568 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:42:14.920508    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e3af0d52-346e-48d7-ac7d-8f2aeee62448-istio-envoy" pod "e3af0d52-346e-48d7-ac7d-8f2aeee62448" (UID: "e3af0d52-346e-48d7-ac7d-8f2aeee62448"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:42:14.920692 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:42:14.920560    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e3af0d52-346e-48d7-ac7d-8f2aeee62448-istio-token" pod "e3af0d52-346e-48d7-ac7d-8f2aeee62448" (UID: "e3af0d52-346e-48d7-ac7d-8f2aeee62448"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:42:14.920692 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:42:14.920576    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/28d7688e-c812-4fcc-b223-151368afe767-istio-token" pod "28d7688e-c812-4fcc-b223-151368afe767" (UID: "28d7688e-c812-4fcc-b223-151368afe767"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:42:14.920692 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:42:14.920641    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/28d7688e-c812-4fcc-b223-151368afe767-kube-api-access-bqk9t" pod "28d7688e-c812-4fcc-b223-151368afe767" (UID: "28d7688e-c812-4fcc-b223-151368afe767"). InnerVolumeSpecName "kube-api-access-bqk9t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:42:14.920843 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:42:14.920799    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/acf3c9ea-1f3e-40d5-88cd-2b76d8897517-kube-api-access-rhwcd" pod "acf3c9ea-1f3e-40d5-88cd-2b76d8897517" (UID: "acf3c9ea-1f3e-40d5-88cd-2b76d8897517"). InnerVolumeSpecName "kube-api-access-rhwcd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:42:14.921238 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:42:14.921216    4943 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/28d7688e-c812-4fcc-b223-151368afe767-istio-podinfo" pod "28d7688e-c812-4fcc-b223-151368afe767" (UID: "28d7688e-c812-4fcc-b223-151368afe767"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 13:42:15.014693 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:42:15.014610    4943 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e3af0d52-346e-48d7-ac7d-8f2aeee62448-workload-certs\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:42:15.014693 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:42:15.014643    4943 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rhwcd\" (UniqueName: \"kubernetes.io/projected/acf3c9ea-1f3e-40d5-88cd-2b76d8897517-kube-api-access-rhwcd\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:42:15.014693 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:42:15.014656    4943 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/28d7688e-c812-4fcc-b223-151368afe767-istio-data\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:42:15.014693 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:42:15.014668    4943 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e3af0d52-346e-48d7-ac7d-8f2aeee62448-istio-podinfo\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:42:15.014693 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:42:15.014678    4943 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/acf3c9ea-1f3e-40d5-88cd-2b76d8897517-istio-token\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:42:15.014693 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:42:15.014686    4943 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/acf3c9ea-1f3e-40d5-88cd-2b76d8897517-workload-certs\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:42:15.014693 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:42:15.014695    4943 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/acf3c9ea-1f3e-40d5-88cd-2b76d8897517-istio-podinfo\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:42:15.015026 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:42:15.014702    4943 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e3af0d52-346e-48d7-ac7d-8f2aeee62448-istio-envoy\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:42:15.015026 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:42:15.014709    4943 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/acf3c9ea-1f3e-40d5-88cd-2b76d8897517-istiod-ca-cert\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:42:15.015026 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:42:15.014718    4943 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/28d7688e-c812-4fcc-b223-151368afe767-istio-podinfo\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:42:15.015026 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:42:15.014726    4943 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/acf3c9ea-1f3e-40d5-88cd-2b76d8897517-istio-envoy\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:42:15.015026 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:42:15.014736    4943 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/28d7688e-c812-4fcc-b223-151368afe767-istiod-ca-cert\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:42:15.015026 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:42:15.014747    4943 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e3af0d52-346e-48d7-ac7d-8f2aeee62448-istio-data\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:42:15.015026 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:42:15.014755    4943 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/acf3c9ea-1f3e-40d5-88cd-2b76d8897517-workload-socket\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:42:15.015026 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:42:15.014763    4943 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/28d7688e-c812-4fcc-b223-151368afe767-workload-socket\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:42:15.015026 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:42:15.014771    4943 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/28d7688e-c812-4fcc-b223-151368afe767-workload-certs\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:42:15.015026 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:42:15.014779    4943 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e3af0d52-346e-48d7-ac7d-8f2aeee62448-istiod-ca-cert\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:42:15.015026 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:42:15.014787    4943 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e3af0d52-346e-48d7-ac7d-8f2aeee62448-credential-socket\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:42:15.015026 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:42:15.014795    4943 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e3af0d52-346e-48d7-ac7d-8f2aeee62448-workload-socket\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:42:15.015026 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:42:15.014802    4943 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e3af0d52-346e-48d7-ac7d-8f2aeee62448-istio-token\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:42:15.015026 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:42:15.014814    4943 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/acf3c9ea-1f3e-40d5-88cd-2b76d8897517-credential-socket\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:42:15.015026 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:42:15.014825    4943 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/28d7688e-c812-4fcc-b223-151368afe767-istio-envoy\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:42:15.015026 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:42:15.014833    4943 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/28d7688e-c812-4fcc-b223-151368afe767-credential-socket\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:42:15.015026 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:42:15.014841    4943 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-55tg9\" (UniqueName: \"kubernetes.io/projected/e3af0d52-346e-48d7-ac7d-8f2aeee62448-kube-api-access-55tg9\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:42:15.015026 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:42:15.014849    4943 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bqk9t\" (UniqueName: \"kubernetes.io/projected/28d7688e-c812-4fcc-b223-151368afe767-kube-api-access-bqk9t\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:42:15.015026 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:42:15.014857    4943 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/28d7688e-c812-4fcc-b223-151368afe767-istio-token\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:42:15.015026 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:42:15.014866    4943 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/acf3c9ea-1f3e-40d5-88cd-2b76d8897517-istio-data\") on node \"ip-10-0-142-84.ec2.internal\" DevicePath \"\""
Oct 30 13:42:15.478250 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:42:15.478213    4943 generic.go:356] "Generic (PLEG): container finished" podID="28d7688e-c812-4fcc-b223-151368afe767" containerID="c2ea5fd9167d5f62257af2c8b088a96935f2de64527cc69155eed73c02633fc6" exitCode=0
Oct 30 13:42:15.478464 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:42:15.478278    4943 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-pd5gs"
Oct 30 13:42:15.478464 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:42:15.478310    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-pd5gs" event={"ID":"28d7688e-c812-4fcc-b223-151368afe767","Type":"ContainerDied","Data":"c2ea5fd9167d5f62257af2c8b088a96935f2de64527cc69155eed73c02633fc6"}
Oct 30 13:42:15.478464 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:42:15.478372    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-pd5gs" event={"ID":"28d7688e-c812-4fcc-b223-151368afe767","Type":"ContainerDied","Data":"e6a8bf040d4a4999834b7481e3b2af7326e9b4b6e4c226a125309c5d05170595"}
Oct 30 13:42:15.478464 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:42:15.478402    4943 scope.go:122] "RemoveContainer" containerID="c2ea5fd9167d5f62257af2c8b088a96935f2de64527cc69155eed73c02633fc6"
Oct 30 13:42:15.479764 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:42:15.479738    4943 generic.go:356] "Generic (PLEG): container finished" podID="883f1e32-141f-45e4-9f0e-31f9231fab0d" containerID="6c1e5ecd734fa6354f717ef54dd3d773e0452a91a0e6fba515a37fea89df216a" exitCode=0
Oct 30 13:42:15.479885 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:42:15.479788    4943 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-q58qm"
Oct 30 13:42:15.479885 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:42:15.479832    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-q58qm" event={"ID":"883f1e32-141f-45e4-9f0e-31f9231fab0d","Type":"ContainerDied","Data":"6c1e5ecd734fa6354f717ef54dd3d773e0452a91a0e6fba515a37fea89df216a"}
Oct 30 13:42:15.480001 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:42:15.479883    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-q58qm" event={"ID":"883f1e32-141f-45e4-9f0e-31f9231fab0d","Type":"ContainerDied","Data":"06d4c642e24ddb4d221354f0b1445d546a1e981e08e6c6f08c5eb3bcdd2abe80"}
Oct 30 13:42:15.481252 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:42:15.481223    4943 generic.go:356] "Generic (PLEG): container finished" podID="e3af0d52-346e-48d7-ac7d-8f2aeee62448" containerID="0033fcc20732bb67a1bf7c2e2cf5e50511434b88f23ad93d8bff99cde7b389c6" exitCode=0
Oct 30 13:42:15.481371 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:42:15.481289    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-xs2vr" event={"ID":"e3af0d52-346e-48d7-ac7d-8f2aeee62448","Type":"ContainerDied","Data":"0033fcc20732bb67a1bf7c2e2cf5e50511434b88f23ad93d8bff99cde7b389c6"}
Oct 30 13:42:15.481371 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:42:15.481304    4943 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-xs2vr"
Oct 30 13:42:15.481371 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:42:15.481361    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-xs2vr" event={"ID":"e3af0d52-346e-48d7-ac7d-8f2aeee62448","Type":"ContainerDied","Data":"d94573e34d6a6969a95d1141f060bf93a358160c7e7009cfa698f6411a5267fb"}
Oct 30 13:42:15.482838 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:42:15.482813    4943 generic.go:356] "Generic (PLEG): container finished" podID="acf3c9ea-1f3e-40d5-88cd-2b76d8897517" containerID="5b917183faf75df89f208e6d23a90615443ffea090d4ed8d6b13b68ec8ef840d" exitCode=0
Oct 30 13:42:15.482952 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:42:15.482858    4943 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-6h9t4"
Oct 30 13:42:15.482952 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:42:15.482886    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-6h9t4" event={"ID":"acf3c9ea-1f3e-40d5-88cd-2b76d8897517","Type":"ContainerDied","Data":"5b917183faf75df89f208e6d23a90615443ffea090d4ed8d6b13b68ec8ef840d"}
Oct 30 13:42:15.482952 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:42:15.482934    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-6h9t4" event={"ID":"acf3c9ea-1f3e-40d5-88cd-2b76d8897517","Type":"ContainerDied","Data":"4e06d74fb0d69f5c68a00d3e322bb7f6374612fdd257675fc3f73bb9b118414c"}
Oct 30 13:42:15.490434 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:42:15.490415    4943 scope.go:122] "RemoveContainer" containerID="c2ea5fd9167d5f62257af2c8b088a96935f2de64527cc69155eed73c02633fc6"
Oct 30 13:42:15.490745 ip-10-0-142-84 kubenswrapper[4943]: E1030 13:42:15.490724    4943 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c2ea5fd9167d5f62257af2c8b088a96935f2de64527cc69155eed73c02633fc6\": container with ID starting with c2ea5fd9167d5f62257af2c8b088a96935f2de64527cc69155eed73c02633fc6 not found: ID does not exist" containerID="c2ea5fd9167d5f62257af2c8b088a96935f2de64527cc69155eed73c02633fc6"
Oct 30 13:42:15.490847 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:42:15.490758    4943 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c2ea5fd9167d5f62257af2c8b088a96935f2de64527cc69155eed73c02633fc6"} err="failed to get container status \"c2ea5fd9167d5f62257af2c8b088a96935f2de64527cc69155eed73c02633fc6\": rpc error: code = NotFound desc = could not find container \"c2ea5fd9167d5f62257af2c8b088a96935f2de64527cc69155eed73c02633fc6\": container with ID starting with c2ea5fd9167d5f62257af2c8b088a96935f2de64527cc69155eed73c02633fc6 not found: ID does not exist"
Oct 30 13:42:15.490847 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:42:15.490779    4943 scope.go:122] "RemoveContainer" containerID="6c1e5ecd734fa6354f717ef54dd3d773e0452a91a0e6fba515a37fea89df216a"
Oct 30 13:42:15.517894 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:42:15.517868    4943 scope.go:122] "RemoveContainer" containerID="6c1e5ecd734fa6354f717ef54dd3d773e0452a91a0e6fba515a37fea89df216a"
Oct 30 13:42:15.518235 ip-10-0-142-84 kubenswrapper[4943]: E1030 13:42:15.518214    4943 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6c1e5ecd734fa6354f717ef54dd3d773e0452a91a0e6fba515a37fea89df216a\": container with ID starting with 6c1e5ecd734fa6354f717ef54dd3d773e0452a91a0e6fba515a37fea89df216a not found: ID does not exist" containerID="6c1e5ecd734fa6354f717ef54dd3d773e0452a91a0e6fba515a37fea89df216a"
Oct 30 13:42:15.518342 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:42:15.518245    4943 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6c1e5ecd734fa6354f717ef54dd3d773e0452a91a0e6fba515a37fea89df216a"} err="failed to get container status \"6c1e5ecd734fa6354f717ef54dd3d773e0452a91a0e6fba515a37fea89df216a\": rpc error: code = NotFound desc = could not find container \"6c1e5ecd734fa6354f717ef54dd3d773e0452a91a0e6fba515a37fea89df216a\": container with ID starting with 6c1e5ecd734fa6354f717ef54dd3d773e0452a91a0e6fba515a37fea89df216a not found: ID does not exist"
Oct 30 13:42:15.518342 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:42:15.518299    4943 scope.go:122] "RemoveContainer" containerID="0033fcc20732bb67a1bf7c2e2cf5e50511434b88f23ad93d8bff99cde7b389c6"
Oct 30 13:42:15.527360 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:42:15.527336    4943 scope.go:122] "RemoveContainer" containerID="0033fcc20732bb67a1bf7c2e2cf5e50511434b88f23ad93d8bff99cde7b389c6"
Oct 30 13:42:15.527701 ip-10-0-142-84 kubenswrapper[4943]: E1030 13:42:15.527680    4943 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0033fcc20732bb67a1bf7c2e2cf5e50511434b88f23ad93d8bff99cde7b389c6\": container with ID starting with 0033fcc20732bb67a1bf7c2e2cf5e50511434b88f23ad93d8bff99cde7b389c6 not found: ID does not exist" containerID="0033fcc20732bb67a1bf7c2e2cf5e50511434b88f23ad93d8bff99cde7b389c6"
Oct 30 13:42:15.527773 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:42:15.527711    4943 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0033fcc20732bb67a1bf7c2e2cf5e50511434b88f23ad93d8bff99cde7b389c6"} err="failed to get container status \"0033fcc20732bb67a1bf7c2e2cf5e50511434b88f23ad93d8bff99cde7b389c6\": rpc error: code = NotFound desc = could not find container \"0033fcc20732bb67a1bf7c2e2cf5e50511434b88f23ad93d8bff99cde7b389c6\": container with ID starting with 0033fcc20732bb67a1bf7c2e2cf5e50511434b88f23ad93d8bff99cde7b389c6 not found: ID does not exist"
Oct 30 13:42:15.527773 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:42:15.527731    4943 scope.go:122] "RemoveContainer" containerID="5b917183faf75df89f208e6d23a90615443ffea090d4ed8d6b13b68ec8ef840d"
Oct 30 13:42:15.536678 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:42:15.536653    4943 scope.go:122] "RemoveContainer" containerID="5b917183faf75df89f208e6d23a90615443ffea090d4ed8d6b13b68ec8ef840d"
Oct 30 13:42:15.536929 ip-10-0-142-84 kubenswrapper[4943]: E1030 13:42:15.536907    4943 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5b917183faf75df89f208e6d23a90615443ffea090d4ed8d6b13b68ec8ef840d\": container with ID starting with 5b917183faf75df89f208e6d23a90615443ffea090d4ed8d6b13b68ec8ef840d not found: ID does not exist" containerID="5b917183faf75df89f208e6d23a90615443ffea090d4ed8d6b13b68ec8ef840d"
Oct 30 13:42:15.537006 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:42:15.536938    4943 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5b917183faf75df89f208e6d23a90615443ffea090d4ed8d6b13b68ec8ef840d"} err="failed to get container status \"5b917183faf75df89f208e6d23a90615443ffea090d4ed8d6b13b68ec8ef840d\": rpc error: code = NotFound desc = could not find container \"5b917183faf75df89f208e6d23a90615443ffea090d4ed8d6b13b68ec8ef840d\": container with ID starting with 5b917183faf75df89f208e6d23a90615443ffea090d4ed8d6b13b68ec8ef840d not found: ID does not exist"
Oct 30 13:42:15.588252 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:42:15.588207    4943 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-xs2vr"]
Oct 30 13:42:15.602107 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:42:15.602063    4943 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-xs2vr"]
Oct 30 13:42:15.630511 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:42:15.630477    4943 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-6h9t4"]
Oct 30 13:42:15.632791 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:42:15.632762    4943 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-6h9t4"]
Oct 30 13:42:15.670677 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:42:15.670639    4943 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-pd5gs"]
Oct 30 13:42:15.677708 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:42:15.677676    4943 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-pd5gs"]
Oct 30 13:42:15.716664 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:42:15.716626    4943 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-q58qm"]
Oct 30 13:42:15.729388 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:42:15.729314    4943 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-q58qm"]
Oct 30 13:42:16.357817 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:42:16.357770    4943 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="28d7688e-c812-4fcc-b223-151368afe767" path="/var/lib/kubelet/pods/28d7688e-c812-4fcc-b223-151368afe767/volumes"
Oct 30 13:42:16.358440 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:42:16.358418    4943 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="883f1e32-141f-45e4-9f0e-31f9231fab0d" path="/var/lib/kubelet/pods/883f1e32-141f-45e4-9f0e-31f9231fab0d/volumes"
Oct 30 13:42:16.358914 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:42:16.358893    4943 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="acf3c9ea-1f3e-40d5-88cd-2b76d8897517" path="/var/lib/kubelet/pods/acf3c9ea-1f3e-40d5-88cd-2b76d8897517/volumes"
Oct 30 13:42:16.359681 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:42:16.359659    4943 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e3af0d52-346e-48d7-ac7d-8f2aeee62448" path="/var/lib/kubelet/pods/e3af0d52-346e-48d7-ac7d-8f2aeee62448/volumes"
Oct 30 13:42:16.647243 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:42:16.647160    4943 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-78df7c7756-pzxsg_bba72dc9-d1a2-4cf5-a205-4ee7250c7916/authorino/0.log"
Oct 30 13:42:16.748586 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:42:16.748549    4943 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-68d6dd76dc-8fv2z_911d5399-ef35-4341-adb3-5d3de9c7b5fb/manager/0.log"
Oct 30 13:42:17.282979 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:42:17.282949    4943 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-7c9b87d574-hdmrm_306df0eb-65a9-4f3f-b55b-fb4389c208e8/limitador/0.log"
Oct 30 13:42:17.456192 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:42:17.456161    4943 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-856dd584c5-hlq89_a303f0b6-1d9d-46a1-9309-43e09bec7618/manager/0.log"
Oct 30 13:42:19.134066 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:42:19.134025    4943 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-must-gather-sstkl/perf-node-gather-daemonset-bfjz8"]
Oct 30 13:42:19.134484 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:42:19.134470    4943 cpu_manager.go:414] "containerMap: removing container" podUID="883f1e32-141f-45e4-9f0e-31f9231fab0d" containerName="istio-proxy"
Oct 30 13:42:19.134543 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:42:19.134486    4943 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="883f1e32-141f-45e4-9f0e-31f9231fab0d" containerName="istio-proxy"
Oct 30 13:42:19.134543 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:42:19.134506    4943 cpu_manager.go:414] "containerMap: removing container" podUID="e3af0d52-346e-48d7-ac7d-8f2aeee62448" containerName="istio-proxy"
Oct 30 13:42:19.134543 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:42:19.134513    4943 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e3af0d52-346e-48d7-ac7d-8f2aeee62448" containerName="istio-proxy"
Oct 30 13:42:19.134543 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:42:19.134529    4943 cpu_manager.go:414] "containerMap: removing container" podUID="acf3c9ea-1f3e-40d5-88cd-2b76d8897517" containerName="istio-proxy"
Oct 30 13:42:19.134543 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:42:19.134540    4943 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="acf3c9ea-1f3e-40d5-88cd-2b76d8897517" containerName="istio-proxy"
Oct 30 13:42:19.134693 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:42:19.134558    4943 cpu_manager.go:414] "containerMap: removing container" podUID="28d7688e-c812-4fcc-b223-151368afe767" containerName="istio-proxy"
Oct 30 13:42:19.134693 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:42:19.134563    4943 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="28d7688e-c812-4fcc-b223-151368afe767" containerName="istio-proxy"
Oct 30 13:42:19.134693 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:42:19.134633    4943 memory_manager.go:365] "RemoveStaleState removing state" podUID="acf3c9ea-1f3e-40d5-88cd-2b76d8897517" containerName="istio-proxy"
Oct 30 13:42:19.134693 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:42:19.134645    4943 memory_manager.go:365] "RemoveStaleState removing state" podUID="28d7688e-c812-4fcc-b223-151368afe767" containerName="istio-proxy"
Oct 30 13:42:19.134693 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:42:19.134654    4943 memory_manager.go:365] "RemoveStaleState removing state" podUID="883f1e32-141f-45e4-9f0e-31f9231fab0d" containerName="istio-proxy"
Oct 30 13:42:19.134693 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:42:19.134662    4943 memory_manager.go:365] "RemoveStaleState removing state" podUID="e3af0d52-346e-48d7-ac7d-8f2aeee62448" containerName="istio-proxy"
Oct 30 13:42:19.139755 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:42:19.139738    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-sstkl/perf-node-gather-daemonset-bfjz8"
Oct 30 13:42:19.160442 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:42:19.160418    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-sstkl/perf-node-gather-daemonset-bfjz8"]
Oct 30 13:42:19.242656 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:42:19.242625    4943 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-sstkl\"/\"default-dockercfg-57nhp\""
Oct 30 13:42:19.246374 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:42:19.246352    4943 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-sstkl\"/\"openshift-service-ca.crt\""
Oct 30 13:42:19.246504 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:42:19.246389    4943 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-sstkl\"/\"kube-root-ca.crt\""
Oct 30 13:42:19.246504 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:42:19.246466    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z888x\" (UniqueName: \"kubernetes.io/projected/0f150a1b-116a-4822-9ed2-1ef02c53b4e1-kube-api-access-z888x\") pod \"perf-node-gather-daemonset-bfjz8\" (UID: \"0f150a1b-116a-4822-9ed2-1ef02c53b4e1\") " pod="openshift-must-gather-sstkl/perf-node-gather-daemonset-bfjz8"
Oct 30 13:42:19.246598 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:42:19.246503    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/0f150a1b-116a-4822-9ed2-1ef02c53b4e1-lib-modules\") pod \"perf-node-gather-daemonset-bfjz8\" (UID: \"0f150a1b-116a-4822-9ed2-1ef02c53b4e1\") " pod="openshift-must-gather-sstkl/perf-node-gather-daemonset-bfjz8"
Oct 30 13:42:19.246598 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:42:19.246542    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/0f150a1b-116a-4822-9ed2-1ef02c53b4e1-podres\") pod \"perf-node-gather-daemonset-bfjz8\" (UID: \"0f150a1b-116a-4822-9ed2-1ef02c53b4e1\") " pod="openshift-must-gather-sstkl/perf-node-gather-daemonset-bfjz8"
Oct 30 13:42:19.246672 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:42:19.246615    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0f150a1b-116a-4822-9ed2-1ef02c53b4e1-sys\") pod \"perf-node-gather-daemonset-bfjz8\" (UID: \"0f150a1b-116a-4822-9ed2-1ef02c53b4e1\") " pod="openshift-must-gather-sstkl/perf-node-gather-daemonset-bfjz8"
Oct 30 13:42:19.246672 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:42:19.246649    4943 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/0f150a1b-116a-4822-9ed2-1ef02c53b4e1-proc\") pod \"perf-node-gather-daemonset-bfjz8\" (UID: \"0f150a1b-116a-4822-9ed2-1ef02c53b4e1\") " pod="openshift-must-gather-sstkl/perf-node-gather-daemonset-bfjz8"
Oct 30 13:42:19.347568 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:42:19.347539    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z888x\" (UniqueName: \"kubernetes.io/projected/0f150a1b-116a-4822-9ed2-1ef02c53b4e1-kube-api-access-z888x\") pod \"perf-node-gather-daemonset-bfjz8\" (UID: \"0f150a1b-116a-4822-9ed2-1ef02c53b4e1\") " pod="openshift-must-gather-sstkl/perf-node-gather-daemonset-bfjz8"
Oct 30 13:42:19.347719 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:42:19.347569    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/0f150a1b-116a-4822-9ed2-1ef02c53b4e1-lib-modules\") pod \"perf-node-gather-daemonset-bfjz8\" (UID: \"0f150a1b-116a-4822-9ed2-1ef02c53b4e1\") " pod="openshift-must-gather-sstkl/perf-node-gather-daemonset-bfjz8"
Oct 30 13:42:19.347719 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:42:19.347592    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/0f150a1b-116a-4822-9ed2-1ef02c53b4e1-podres\") pod \"perf-node-gather-daemonset-bfjz8\" (UID: \"0f150a1b-116a-4822-9ed2-1ef02c53b4e1\") " pod="openshift-must-gather-sstkl/perf-node-gather-daemonset-bfjz8"
Oct 30 13:42:19.347719 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:42:19.347626    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0f150a1b-116a-4822-9ed2-1ef02c53b4e1-sys\") pod \"perf-node-gather-daemonset-bfjz8\" (UID: \"0f150a1b-116a-4822-9ed2-1ef02c53b4e1\") " pod="openshift-must-gather-sstkl/perf-node-gather-daemonset-bfjz8"
Oct 30 13:42:19.347719 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:42:19.347662    4943 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/0f150a1b-116a-4822-9ed2-1ef02c53b4e1-proc\") pod \"perf-node-gather-daemonset-bfjz8\" (UID: \"0f150a1b-116a-4822-9ed2-1ef02c53b4e1\") " pod="openshift-must-gather-sstkl/perf-node-gather-daemonset-bfjz8"
Oct 30 13:42:19.347719 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:42:19.347696    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/0f150a1b-116a-4822-9ed2-1ef02c53b4e1-lib-modules\") pod \"perf-node-gather-daemonset-bfjz8\" (UID: \"0f150a1b-116a-4822-9ed2-1ef02c53b4e1\") " pod="openshift-must-gather-sstkl/perf-node-gather-daemonset-bfjz8"
Oct 30 13:42:19.347903 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:42:19.347698    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/0f150a1b-116a-4822-9ed2-1ef02c53b4e1-podres\") pod \"perf-node-gather-daemonset-bfjz8\" (UID: \"0f150a1b-116a-4822-9ed2-1ef02c53b4e1\") " pod="openshift-must-gather-sstkl/perf-node-gather-daemonset-bfjz8"
Oct 30 13:42:19.347903 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:42:19.347738    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/0f150a1b-116a-4822-9ed2-1ef02c53b4e1-proc\") pod \"perf-node-gather-daemonset-bfjz8\" (UID: \"0f150a1b-116a-4822-9ed2-1ef02c53b4e1\") " pod="openshift-must-gather-sstkl/perf-node-gather-daemonset-bfjz8"
Oct 30 13:42:19.347903 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:42:19.347755    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0f150a1b-116a-4822-9ed2-1ef02c53b4e1-sys\") pod \"perf-node-gather-daemonset-bfjz8\" (UID: \"0f150a1b-116a-4822-9ed2-1ef02c53b4e1\") " pod="openshift-must-gather-sstkl/perf-node-gather-daemonset-bfjz8"
Oct 30 13:42:19.373229 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:42:19.373196    4943 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z888x\" (UniqueName: \"kubernetes.io/projected/0f150a1b-116a-4822-9ed2-1ef02c53b4e1-kube-api-access-z888x\") pod \"perf-node-gather-daemonset-bfjz8\" (UID: \"0f150a1b-116a-4822-9ed2-1ef02c53b4e1\") " pod="openshift-must-gather-sstkl/perf-node-gather-daemonset-bfjz8"
Oct 30 13:42:19.452568 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:42:19.452543    4943 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-sstkl/perf-node-gather-daemonset-bfjz8"
Oct 30 13:42:19.685140 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:42:19.685111    4943 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-884576c57-2sfj9_19b38c46-bfd4-4ed0-a764-f7b672f18cc6/cluster-monitoring-operator/0.log"
Oct 30 13:42:19.757299 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:42:19.757250    4943 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-sstkl/perf-node-gather-daemonset-bfjz8"]
Oct 30 13:42:19.758852 ip-10-0-142-84 kubenswrapper[4943]: W1030 13:42:19.758823    4943 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod0f150a1b_116a_4822_9ed2_1ef02c53b4e1.slice/crio-b6c43de10b7082cc2e569efb0a46d5f5e4e0cb5d284f5762bdcb85bdb02fc67c WatchSource:0}: Error finding container b6c43de10b7082cc2e569efb0a46d5f5e4e0cb5d284f5762bdcb85bdb02fc67c: Status 404 returned error can't find the container with id b6c43de10b7082cc2e569efb0a46d5f5e4e0cb5d284f5762bdcb85bdb02fc67c
Oct 30 13:42:20.518946 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:42:20.518910    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-sstkl/perf-node-gather-daemonset-bfjz8" event={"ID":"0f150a1b-116a-4822-9ed2-1ef02c53b4e1","Type":"ContainerStarted","Data":"9b2add78267649eeb882e003ee22f939ccfe3f22bd22cabcd141c0feb70b427b"}
Oct 30 13:42:20.519390 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:42:20.518969    4943 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-sstkl/perf-node-gather-daemonset-bfjz8" event={"ID":"0f150a1b-116a-4822-9ed2-1ef02c53b4e1","Type":"ContainerStarted","Data":"b6c43de10b7082cc2e569efb0a46d5f5e4e0cb5d284f5762bdcb85bdb02fc67c"}
Oct 30 13:42:20.519390 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:42:20.519007    4943 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-sstkl/perf-node-gather-daemonset-bfjz8"
Oct 30 13:42:20.563965 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:42:20.563921    4943 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-must-gather-sstkl/perf-node-gather-daemonset-bfjz8" podStartSLOduration=1.5639064870000001 podStartE2EDuration="1.563906487s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 13:42:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:42:20.562529885 +0000 UTC m=+2716.350533616" watchObservedRunningTime="2026-10-30 13:42:20.563906487 +0000 UTC m=+2716.351910226"
Oct 30 13:42:21.117949 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:42:21.117920    4943 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-slf7p_ca443547-3eb9-4f7a-b85f-f4e1c87ea770/node-exporter/0.log"
Oct 30 13:42:21.236381 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:42:21.236355    4943 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-slf7p_ca443547-3eb9-4f7a-b85f-f4e1c87ea770/kube-rbac-proxy/0.log"
Oct 30 13:42:21.352912 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:42:21.352885    4943 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-slf7p_ca443547-3eb9-4f7a-b85f-f4e1c87ea770/init-textfile/0.log"
Oct 30 13:42:25.528224 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:42:25.528188    4943 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-86c478bb88-vsqgb_4883c26d-0877-4870-b0ee-406f5bfa8cf4/networking-console-plugin/0.log"
Oct 30 13:42:26.534607 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:42:26.534581    4943 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-sstkl/perf-node-gather-daemonset-bfjz8"
Oct 30 13:42:26.593905 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:42:26.593866    4943 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-8fqfr_d8930ee7-0ae0-48b5-8ce3-cb8820355cef/console-operator/2.log"
Oct 30 13:42:26.598347 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:42:26.598321    4943 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-8fqfr_d8930ee7-0ae0-48b5-8ce3-cb8820355cef/console-operator/3.log"
Oct 30 13:42:28.963092 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:42:28.963064    4943 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-58c9f5f46d-xtfpg_f07abb3f-8900-4fa5-8d35-c7ab1d35582d/volume-data-source-validator/0.log"
Oct 30 13:42:31.631448 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:42:31.631419    4943 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-g2ckh_fcf99170-f5cf-4987-97bd-1c30b1b37a35/dns/0.log"
Oct 30 13:42:31.760402 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:42:31.760373    4943 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-g2ckh_fcf99170-f5cf-4987-97bd-1c30b1b37a35/kube-rbac-proxy/0.log"
Oct 30 13:42:32.205366 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:42:32.205336    4943 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-4lfbk_eca286e8-0b02-42af-9d15-1d8670b7362c/dns-node-resolver/0.log"
Oct 30 13:42:32.480657 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:42:32.480579    4943 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-7ddd9cfdc8-vvkvh_1d6ac649-474a-4e72-b202-259df21727a5/discovery/0.log"
Oct 30 13:42:32.522631 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:42:32.522607    4943 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-75fdd59bb7-cpmzt_e4cc43ab-deda-4426-83f5-236f95bfdfc4/kube-auth-proxy/0.log"
Oct 30 13:42:32.712220 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:42:32.712186    4943 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7b98bb678c-666x8_94a883ed-7c63-402d-bf2e-2e4ce2383175/payload-pre-processing/0.log"
Oct 30 13:42:32.774810 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:42:32.774741    4943 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5f78687d5b-r2pcb_7d0643d5-c375-42a2-ab82-f90aa05ec22d/payload-processing/0.log"
Oct 30 13:42:32.839228 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:42:32.839197    4943 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-649bdc978f-lkpqz_e1f629e6-6a96-444f-8329-e71bb98c9029/router/0.log"
Oct 30 13:42:33.697666 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:42:33.697635    4943 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-h8q28_16402666-f516-49d1-be0d-f6ab7234e637/node-ca/0.log"
Oct 30 13:42:35.959630 ip-10-0-142-84 kubenswrapper[4943]: E1030 13:42:35.959591    4943 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-knqz6" containerName="extractor"
Oct 30 13:42:36.133764 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:42:36.133728    4943 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-7ddd9cfdc8-vvkvh_1d6ac649-474a-4e72-b202-259df21727a5/discovery/0.log"
Oct 30 13:42:36.195947 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:42:36.195921    4943 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-75fdd59bb7-cpmzt_e4cc43ab-deda-4426-83f5-236f95bfdfc4/kube-auth-proxy/0.log"
Oct 30 13:42:36.413563 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:42:36.413495    4943 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7b98bb678c-666x8_94a883ed-7c63-402d-bf2e-2e4ce2383175/payload-pre-processing/0.log"
Oct 30 13:42:36.478362 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:42:36.478333    4943 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5f78687d5b-r2pcb_7d0643d5-c375-42a2-ab82-f90aa05ec22d/payload-processing/0.log"
Oct 30 13:42:36.549952 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:42:36.549923    4943 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-649bdc978f-lkpqz_e1f629e6-6a96-444f-8329-e71bb98c9029/router/0.log"
Oct 30 13:42:37.154395 ip-10-0-142-84 kubenswrapper[4943]: I1030 13:42:37.154364    4943 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-7ddd9cfdc8-vvkvh_1d6ac649-474a-4e72-b202-259df21727a5/discovery/0.log"